Magento Index Seminar Video & Guide

Last week there was an online seminar about Magento Indexes functionality. In this blog post I would like to describe a bit more properly a solution for custom index creation, because there were some things that were some issues with example in the slides. Just was a last moment change before seminar to make it multi-store instead of global.

Custom Indexer Creation Guide

Here you can see step by step implementation of a possible example with featured products indexer. As I told on seminar there is more easy and faster way to create featured products, for instance if you add “is_featured” to flat index. But this one is good for learning simple indexer creation.

You can download a full module code by clicking here.

1. Creating Product Attribute

Let’s create a product attribute, that will be used for marking our products as featured.

$this->addAttribute(Mage_Catalog_Model_Product::ENTITY, 'is_featured', array(
    'type' => 'int',
    'label' => 'Is featured',
    'input' => 'select',
    'source' => 'eav/entity_attribute_source_boolean',
    'global' => Mage_Catalog_Model_Resource_Eav_Attribute::SCOPE_STORE,
    'user_defined' => false,
    'required' => false
));

So as you see in the code, it has store view scope for making possible to define different featured products for every store view.

2. Designing Index Table

Before you implement  a new indexer you need to take care about how the index table will look like. E.g. you need to create  a table structure that will be optimized for using it on the frontend. So we are going to create a table with only two columns: store_id and product_id. Of course let’s use available Varien_Db_Ddl_Table.

// Defining table structure via object
$table = $this->getConnection()
    ->newTable($this->getTable('ecomdev_indexseminar/featured_products'))
    ->addColumn('store_id', Varien_Db_Ddl_Table::TYPE_SMALLINT, null,
                array(
                    'unsigned' => true,
                    'nullable' => false,
                    'primary' => true
                ))
    ->addColumn('product_id', Varien_Db_Ddl_Table::TYPE_INTEGER, null,
                array(
                    'unsigned' => true,
                    'nullable' => false,
                    'primary' => true
                ))
   ->addForeignKey(
        $this->getFkName('ecomdev_indexseminar/featured_products', 'store_id', 'core/store', 'store_id'),
        'store_id', $this->getTable('core/store'), 'store_id',
        Varien_Db_Ddl_Table::ACTION_CASCADE, Varien_Db_Ddl_Table::ACTION_CASCADE
    )
    ->addForeignKey(
        $this->getFkName('ecomdev_indexseminar/featured_products', 'product_id', 'catalog/product', 'entity_id'),
        'product_id', $this->getTable('catalog/product'), 'entity_id',
        Varien_Db_Ddl_Table::ACTION_CASCADE, Varien_Db_Ddl_Table::ACTION_CASCADE
    );
// Executing table creation object
$this->getConnection()->createTable($table);

Foreign keys just used for auto-cleaning the index table if product or store view was deleted. You may not add them if you need to have additional performance on index creation process.

3. Defining Indexer in Configuration

So now we going to proceed with creation indexer model itself, so let’s define it in configuration, to let Magento know about new indexer.

<config>
  <!-- other code... -->
  <global>
    <!-- other code... -->
        <index>
            <indexer>
                <featured_products>
                    <model>ecomdev_indexseminar/indexer_featured</model>
                 </featured_products>
            </indexer>
        </index>
    <!-- other code... -->
  </global>
 <!-- other code... -->
</config>

So we need just to specify our indexer code that can be used for getProcessByCode() method and model name that will realize our indexer itself. In this code snipped, indexer code is “featured_products” and model is
“ecomdev_indexseminar/indexer_featured”.

4. Creating Indexer Model

All the indexer models should be extended from Mage_Index_Model_Indexer_Abstract.

class EcomDev_IndexSeminar_Model_Indexer_Featured
    extends Mage_Index_Model_Indexer_Abstract
{
}

Then you need to define events that this indexer should process by _registerEvent method, i.e. matching entity and event types to make them available for index.

class EcomDev_IndexSeminar_Model_Indexer_Featured
    extends Mage_Index_Model_Indexer_Abstract
{
    protected $_matchedEntities = array(
        Mage_Catalog_Model_Product::ENTITY => array(
            Mage_Index_Model_Event::TYPE_SAVE,
            Mage_Index_Model_Event::TYPE_MASS_ACTION
        )
    );
}

In our case it is only save and mass product update events for catalog product.

Also lets create _construct method, that initializes our resource model (our database layer for indexer)

class EcomDev_IndexSeminar_Model_Indexer_Featured
    extends Mage_Index_Model_Indexer_Abstract
{
// .. previous lines
    protected function _construct()
    {
        $this->_init('ecomdev_indexseminar/indexer_featured');
    }
}

Then we need to add methods that will return description and our indexer name for displaying on “System -> Index Management” page. They are quite simple doesn’t require to explanation.

class EcomDev_IndexSeminar_Model_Indexer_Featured
    extends Mage_Index_Model_Indexer_Abstract
{
// .. previous lines
    public function getName()
    {
        return Mage::helper('ecomdev_indexseminar')->__('Featured Product');
    }
    public function getDescription()
    {
        return Mage::helper('ecomdev_indexseminar')->__('Indexes something');
    }
}

Then we need to realize a method that will perform a check for changes in product, that may affect rebuilding index and store that data for our indexer further process.

class EcomDev_IndexSeminar_Model_Indexer_Featured
    extends Mage_Index_Model_Indexer_Abstract
{
// .. previous lines
    protected function _registerEvent(Mage_Index_Model_Event $event)
    {
        /* @var $entity Mage_Catalog_Model_Product */
        $entity = $event->getDataObject();
        if ($entity->dataHasChangedFor('is_featured')) {
            $event->setData('product_id', $entity->getId());
        } elseif ($entity->getAttributesData()) {
            $attributeData = $entity->getAttributesData();
            if (isset($attributeData['is_featured'])) {
                $event->setData('product_ids', $entity->getProductIds());
            }
        }
    }
}

So now let take a look what it actually does:

• If it receives a product model that has “is_featured” attribute value changed, then it stores this product id for event data. Usually indexers add own prefix to values that are stored to event, to not conflict with each other.
• Then it checks if it is a massaction object, that contains a change for “is_featured” attribute as well. If our attribute changed, then store an array of product ids to reindex.
• And of course it doesn’t store anything if related data to its logic was not changed

Okey, seems our indexer registers proper data for further indexing process, so let go forward. Now it’s time to create method that runs indexer resource model method depending on entity and event types.

class EcomDev_IndexSeminar_Model_Indexer_Featured
    extends Mage_Index_Model_Indexer_Abstract
{
// .. previous lines
    protected function _processEvent(Mage_Index_Model_Event $event)
    {
        if ($event->getData('product_id') || $event->getData('product_ids')) {
            $this->callEventHandler($event);
        }
    }
}

So this method only checks if there is any data we set before, and uses callEventHandler() method that invokes proper resource model method. For instance if we have entity type “catalog_product” and event type “save” then the name of invoked method will be “catalogProductSave”, i.e. concatenated and “camelized” name from this two properties of event.

5. Creating Resource Model

So the index model is ready, the only one most important part left. It is our index model that generates all the data for index. Index module also has an abstract class for that we should extend from: Mage_Index_Model_Resource_Abstract.

class EcomDev_IndexSeminar_Model_Resource_Indexer_Featured
    extends Mage_Index_Model_Resource_Abstract
{
    protected function _construct()
    {
        // Defining the connections prefix for our resource.
        $this->_setResource('ecomdev_indexseminar');
    }
}

Also as you see it does some initialization in _construct method, for letting Magento know wich connection it may use, if there will be different connections for indexer module.

Then we are going to create the main method that will be invoked for rebuilding any kind of index, I made it flexible, because I don’t like copy-paste . The method body is quite good explained by comments in the snippet itself.

class EcomDev_IndexSeminar_Model_Resource_Indexer_Featured
    extends Mage_Index_Model_Resource_Abstract
{
// ... other code
    protected function _reindexEntity($productId = null)
    {
        // New select object for data for index retrieval
        $select = $this->_getWriteAdapter()->select();
        // Getting attribute model for retrieving its meta-information (attribute id, backend table name)
        /* @var $attribute Mage_Catalog_Model_Resource_Eav_Attribute */
        $attribute = Mage::getSingleton('eav/config')
                         ->getAttribute('catalog_product', 'is_featured');
        $select
            // Select all store views
            ->from(array('store' => $this->getTable('core/store')), array('store_id'))
            // Joining default store value
            ->join(array('default_value' => $attribute->getBackendTable()),
                         'default_value.store_id = 0', array('entity_id'))
            // Select current store value
            ->joinLeft(array('store_value' => $attribute->getBackendTable()),
                       'store_value.attribute_id = default_value.attribute_id'
                       . ' AND store_value.entity_id = default_value.entity_id'
                       . ' AND store_value.store_id = store.store_id',
                       array())
            // Limiting select by only featured products
            ->where('default_value.value = ? OR store_value.value = ?', 1)
            // Add condition if current store value is not "No" (empty or not equals to 0)
            ->where('store_value.value IS NULL OR store_value.value != ?', 0)
            // Limiting select only by our attribute data
            ->where('default_value.attribute_id = ?', $attribute->getId())
            // Do not generate values for default store
            ->where('store.store_id != ?', 0);
        // If index should be rebuild by one or some set of products
        if ($productId !== null) {
            // If single product id, then cast to array
            if (!is_array($productId)) {
                $productId = array($productId);
            }
            $select->where('default_value.entity_id IN(?)', $productId);
            // Cleaning existing rows for product to reindex
            $this->_getIndexAdapter()->delete(
                $this->getTable('ecomdev_indexseminar/featured_products'),
                array(
                    'product_id IN(?)' => $productId
                )
            );
        } else { // If it is full re-index clear all index data
            $this->_getIndexAdapter()->delete($this->getTable('ecomdev_indexseminar/featured_products'));
        }
        // Fullfil the index table from our select
        $this->insertFromSelect(
            $select,
            $this->getTable('ecomdev_indexseminar/featured_products'),
            array('store_id', 'product_id')
        );
    }
}

After we have unified method for generation of index data, we can just call it from different index event handlers:

class EcomDev_IndexSeminar_Model_Resource_Indexer_Featured
    extends Mage_Index_Model_Resource_Abstract
{
// ... other code
    // Handler for "Reindex" action in the admin panel or console
    public function reindexAll()
    {
        // Call our reindex method without parameters,
        // to rebuild all data
        $this->_reindexEntity();
    }
    // Handler for save event on particular product
    public function catalogProductSave($event)
    {
        // Passing single product id to reindex method
        $this->_reindexEntity($event->getData('product_id'));
    }
    // Handler for updating products data via massaction
    public function catalogProductMassAction($event)
    {
        // Passing an array of product ids to our reindex method
        $this->_reindexEntity($event->getData('product_ids'));
    }
}

So congratulation you have working custom indexer!

To apply this data on the fronend you can find a small example with observer for it in the code, available to download. It observes catalog_product_collection_apply_limitations_after event and calls a method on resource model that joins our index table, so limits selection results.

Also it uses collections flag functionality to add index only when it needed. So if you want to display featured products in the block, you need to set flag “featured” to a product collection, before loading the collection itself.

$collection->setFlag('featured', true);

Seminar Video

Before you are going to check this video recording from seminar, I want to excuse for some issues with the presentation, I just got cold one day before it and it was first such an online seminar when you don’t have visual contact with attendees. But anyway I hope you enjoy it

Magento Index Functionality Seminar – Sep 8 from Ivan Chepurnyi on Vimeo.

Next Seminar

Also we are planning to make it on regular basis. Our goal is to provide at least one seminar a month.
So the next seminar possible topics are presented in the voting form. The most voted topic will be prepared for the one, that will be planned for October. Most probably at the end of the month.

Written by Ivan Chepurnyi

September 16th 2011 - 5 Comments