How does a new feature end up in Joomla?

“The Category Item Counter”

Category Item Counter

by Peter Martin / @pe7er

About me

Peter Martin, Nijmegen, NL
db8.nl - Joomla support, application development

db8 Site Dev - free checklist component for devs

Joomla volunteer:
* Global Forum Moderator
* Joomla Bug Squad
* co-organizing PBFs in NL
* former Joomla CLT
* Mentor GSoC 2016

Likes:
* Music (vinyl records)
* Movies
* Electronics
* Raspberry Pi
* Internet of Things

Organizes:
* Linux Usergroup Nijmegen
* Open Coffee Nijmegen

www.db8.nl
twitter: @pe7er

Overview presentation

  • Origin of Category Item Counter
  • Well kept secret
  • Joomla's development process
  • My workflow
  • More generic?
  • Problems
  • Components learn to count
  • More problems

Origin of this "nice" feature

Joomladay Nice

Saturday 9 May 2015
around 15:00 hours

“It's a pity that the
Joomla Category Manager doesn't have
an Article Counter anymore”

Marc Dechèvre

Lesson 1

Share your ideas
for improvements constructively,
preferably in real life
(and not on Twitter)

Joomla 1.5 - Category Manager

“That should not be
that hard to build in, should it?”

Yours truly

(who had not been to the hairdresser's yet)

Lesson 2

Have fun

(or make it)

deux bières

encore deux bières

Back in the hotel

com_category,
a well-kept secret

Joomla 1.5 & categories

  • Articles (com_content): Articles + Category Manager

  • Banners (com_banners): Banners + Category Manager

  • Contacts (com_contacts): Contacts + Category Manager

  • Newsfeeds (com_newsfeeds): Newsfeeds + Cat Manager

  • Weblinks (com_weblinks): Weblinks + Category Manager

Category Item Counter: items counted with an SQL query

Joomla 3.x & categories

  • Articles (com_content): Articles

  • Banners (com_banners): Banners

  • Contacts (com_contacts): Contacts

  • Newsfeeds (com_newsfeeds): Newsfeeds

  • Weblinks (com_weblinks): Weblinks

  • Categories (com_categories): generic component

Articles & categories

  • Article Manager: Articles
    (Content > Article Manager)
    URL: /administrator/index.php
    ?option=com_content&view=articles

  • Category Manager: Articles
    (Content > Category Manager)
    URL: /administrator/index.php
    ?option=com_categories
    &extension=com_content

Banners & categories

  • Banner Manager: Banners
    (Components > Banners > Banners)
    URL: /administrator/index.php
    ?option=com_banners

  • Category Manager: Banners
    (Components > Banners > Categories)
    URL: /administrator/index.php
    ?option=com_categories
    &extension=com_banners

Use com_categories
in your own components

in administrator/components/
com_yourcomponent/models/forms/some_item.xml

<field
	name="catid"
	type="category"
	extension="com_yourcomponent"
	default=""
	class="inputbox"
	label="JCATEGORY"
	description="JFIELD_CATEGORY_DESC"
	required="true">
		<option value="0">JOPTION_SELECT_CATEGORY</option>
</field>

Lesson 3

Limit the scope
of your project...

Take small steps

My scope

com_categories

<?php
	if($extension == "com_content"):
	//add count SQL query to $query object
?>

Joomla's development process

git

Software Version Control

  • Distributed - no central server
  • Historic overview of code changes

github.com

Software as a Service

github.com

  • PR = Pull Request
    ask Joomla to pull the code of your private fork
    into Joomla's core code

  • RTC = Ready To Commit
    need 3 succesfull tests
    → code maybe in next Joomla version...

Lesson 4

Be prepared to test

Patch Tester Component

  1. Install Joomla staging
    + default testing data
  2. Install Patch Testing Component
  3. load list of Pull Requests
  4. read testing instructions, test
  5. install patch
  6. test, write report
  7. uninstall patch

Joomla releases

Current version: Joomla 3.6.2

Patch to solve bug → next subversion (3.6.3)

New Feature → next major version (3.7)
(labels: “milestone” + “New feature”)

New language string
(label: “new language string”)

My workflow

Create copy (fork)

Clone to PC

Share new code

Lesson 5

Write clear
testing instructions

Get tested

Write clear instructions

Automatic test

Codestyle test

tip: install PHP Code Sniffer in your IDE

2x manual test

Test report via https://issues.joomla.org/

Lesson 6

Use screen dumps!
A picture = thousand words

Before the PR

After the PR

“Hathor template?”
“Alignment looks bad”
“Travis is not happy...
Can you try to fix the Travis errors?”

Almost there...

Foto: Pierre Sempé

...but not quite

like this?

End of Joomladay France 2015

Hathor... Anyone?

Generic = more beautiful

Lesson 7

Dare to ask

Articles (com_content)

  • Article Manager: Articles
    via Content > Article Manager
    URL /administrator/index.php ?option=com_content
    &view=articles

  • Category Manager: Articles
    via Content > Category Manager
    URL /administrator/index.php?option=com_categories
    &extension=com_content

Left menu in Articles

  • Category Manager (com_categories)
    knows via extension=com_content

    and asks for helper file:
    /administrator/components/
    com_content/ helpers/content.php

    the helper file generates the left menu.

Extend $query

  • Category Manager (com_categories)
    administrator/components/
    com_categories/models/categories.php
    contains protected function getListQuery()

    → extend $query object via
    $classname::countItems($query);

    → load specific code from helper file
    of the extension=com_componentnaam component.

HTML output: Column header

  • administrator/components/com_categories/views/
    categories/tmpl/default.php
    <?php if (isset($this->items[0])
    && property_exists($this->items[0],
    'count_published')) :
    $columns++; ?>
    
    <th width="1%" class="nowrap center hidden-phone">
    <i class="icon-publish"></i>
    </th>
    
    <?php endif;?>

HTML output:
clickable counter

  • administrator/components/com_categories/views/
    categories/tmpl/default.php
    <?php if (isset($this->items[0])
    && property_exists($this->items[0], 'count_published')) : ?>
    
    <td class="center btns hidden-phone">
      <a class="badge <?php if ($item->count_published > 0)
      echo "badge-success"; ?>"
      title="<?php echo Jtext::_('COM_ CATEGORY_COUNT_PUBLISHED_ITEMS');?>"
      href="<?php echo JRoute::_('index.php?option=' . $component .
      '&filter[category_id]=' . (int) $item->id .
      '&filter[published]=1' .
      '&filter[level]=' . (int) $item->level);?>">
    
        <?php echo $item->count_published; ?>
    
    </a></td>
    <?php endif;?>

More problems
in the core code

Triggering the Filters

  • Publish state in core components
    &filter[published]


  • Publish state in Banners (com_banners)
    &filter[state]

Teach your own
components to count

Count published items

/administrator/components/com_your_
component
/helpers/your_component.php

class YourcomponentHelper extends JHelperContent
{
  public static function countItems(&$query)
  {
    // Join articles to cats and count published items
    $query->select('COUNT(DISTINCT cp.id) AS count_published');
    $query->join('LEFT', '#__yourcomponent_items AS cp
    ON cp.catid = a.id AND cp.state = 1');

    return $query;
  }
}

* Do NOT use this SQL query... I'll explain in a minute...

Count other states

// Count unpublished items
$query->select('COUNT(DISTINCT cu.id) AS count_unpublished');
$query->join('LEFT', '#__yourcomponent_items AS cu
ON cu.catid = a.id AND cu.state = 0');

// Count archived items
$query->select('COUNT(DISTINCT ca.id) AS count_archived');
$query->join('LEFT', '#__yourcomponent_items AS ca
ON ca.catid = a.id AND ca.state = 2');

// Count trashed items
$query->select('COUNT(DISTINCT ct.id) AS count_trashed');
$query->join('LEFT', '#__yourcomponent_items AS ct
ON ct.catid = a.id AND ct.state = -2');

* Do NOT use this SQL query... I'll explain in a minute NOW:

Joomla, we have a problem

Just before the release of Joomla 3.5.0

Go on, count...

Some sites :-)

But not ours... Our websites are all perfectly orgganizes, aren't they?

Performance problem

“Category Manager is very slow on my site with 100 categories and 85.000 articles” https://github.com/joomla/joomla-cms/pull/6916#issuecomment-195671451

SQL: COUNT(DISTINCT)

“New category count feature performance degrade #9420”
“Database queries total: 46.3 ms > 31,358.46 ms (31 sec!)”
https://github.com/joomla/joomla-cms/pull/9420

Count faster...

https://github.com/joomla/joomla-cms/pull/9421

1. Moved

administrator/components/
com_categories/models/categories.php

// Load Helper file of the component for
// which com_categories displays the categories
$classname = ucfirst(substr($extension, 4)) . 'Helper';

if (class_exists($classname) && method_exists($classname,
'countItems'))
{
  // Get the SQL to extend the com_category $query object
  // with item count (published, unpublished, trashed)
  // $classname::countItems($query);
  $classname::countItems($this->items);
}
to administrator/components/
com_categories/views/categories/tmpl/default.php

2. Removed

administrator/components/
com_content/helpers/content.php

Extending COUNT DISTINCT from $query object removed:

$query->select('COUNT(DISTINCT cp.id) AS count_published');
$query->join('LEFT', '#__content AS cp
ON cp.catid = a.id AND cp.state = 1');

3a. Changed

administrator/components/
com_content/helpers/content.php

public static function countItems(&$items)
{
  $db = JFactory::getDbo();
  foreach ($items as $i => $item)
  {
    $item->count_trashed = 0;
    $item->count_archived = 0;
    $item->count_unpublished = 0;
    $item->count_published = 0;
    $query = $db->getQuery(true);
    $query->select('state, count(*) AS count')
          ->from($db->qn('#__content'))
          ->where('catid = ' . (int) $item->id)
          ->group('state');
    $db->setQuery($query);

3b. Changed

administrator/components/
com_content/helpers/content.php

    $arts = $db->loadObjectList();

    foreach ($arts as $i => $art) {
      if ($art->state == 1) { $item->count_published = $art->count; }

      if ($art->state == 0) {
          $item->count_unpublished = $art->count; }

      if ($art->state == 2) { $item->count_archived = $art->count; }

      if ($art->state == -2) { $item->count_trashed = $art->count; }
    }
  }
return $items;
}

Prob 2: New countItems feature

“only works for basic category calls without sections
because the extension string is not handled properly”

https://github.com/joomla/joomla-cms/pull/9580 (fixed #9186 #9462 )

Summary

Summary

  1. Share your ideas for improvements
    constructively, preferably in real life
    (and not on Twitter)
  2. Have fun (or make it)
  3. Limit the scope of your project... Take small steps
  4. Be prepared to test
  5. Write clear testing instructions
  6. Use screen dumps! A picture = thousand words
  7. Dare to ask

Thanks!

Thanks to everybody that helped developing this new feature!

Peter Martin
e-mail: info at db8.nl
twitter: @pe7er
presentation: https://db8.nl

Credits