Magento Dataflow - standard parsers and mapping values [part 4]

As promised in Magento Dataflow - Default Adapters [Part 2] today I will write about standard parsers in Magento DataFlow module and mapping values with mappers.

1. Parser definition

   Parsers are responsible for transforming data from. Parser's interface Mage_Dataflow_Model_Convert_Parser_Interface defines two methods required in each parser: parse() and unparse(). Definition of parser within profile's xml can be as simple as:

   <actiontype="dataflow/convert_parser_serialize"method="parse"/>

   Similar to adapter we define action tag with two attributes: type, which tells which class we want to use and this class's method we want to call. We can also call parser passing variables within action tag body as you will see below.

2. Standard parsers

   Magento DataFlow includes few standard parsers which you can find in app/code/core/Dataflow/Model/Convert/Parser.

   The simplest of standard parsers is dataflow/convert_parser_serialize (Mage_Dataflow_Model_Convert_Parser_Serialize) which doesn't require any variables passed. It requires though that any of previous actions set data within profile's container. Method parse() unserialize data stored within profile's container and replace it with the result. Method unparse() do the opposite, so it serializes data stored within profile's container and replace it with the result.

   One of most often used standard parsers is dataflow/convert_parser_csv which allows transforming from (with method parse()) or to (with method unparse()) CSV file. Example of definition:

   <actiontype="dataflow/convert_parser_csv"method="parse">
       <varname="delimiter"><![CDATA[,]]></var>
       <varname="enclose"><![CDATA["]]></var>
       <varname="fieldnames">true</var>
       <varname="store"><![CDATA[0]]></var>
       <varname="decimal_separator"><![CDATA[.]]></var>
       <varname="adapter">catalog/convert_adapter_product</var>
       <varname="method">parse</var>
   </action>

   This parser requires that you call some io adapter prior to its execution (using for example dataflow/convert_adapter_io to read some csv file) if you want to call method parse. If you want to store data into CSV file you have to do both - call any action that will set data within profile's container prior to parser execution and call io adapter after parser execution to store data within file.

   Following variables will allow you to customize csv file parsing:

   • delimiter - defines delimiter used in csv file; defaults to comma (,) character
   • enclose - defines what character is used to enclose data values; defaults to empty character
   • escape - defines escape character for csv file; defaults to \\
   • decimal_separator - defines decimal separator sign
   • fieldnames - if set to true, it is assumed first row of csv file contains field names; if set to false map variable is used
   • map - defines fieldnames for files where first row doesn't contain fieldnames; to see how to define a map take a look at section of this article related to mapping values
   • adapter - tells which adapters method should be called on each row
   • method - tells which method of adapter should be called on each row; defaults to saveRow

   All variables defined within parser's action body are passed to the defined adapter, so if you need to pass something to it, you can simply set required variable within parser's action body.

   Last of standard parsers included within DataFlow module is dataflow/convert_parser_xml_excel (Mage_Dataflow_Model_Convert_Parser_Xml_Excel), which converts data from and to Excel xml file. Example of definition:

   <actiontype="dataflow/convert_parser_xml_excel"method="unparse">
       <varname="single_sheet"><![CDATA[products]]></var>
       <varname="fieldnames">true</var>
   </action>

   Use requirements are the same as for dataflow/convert_parser_csv.

   Following variables will allow you to customize csv file parsing:

   • fieldnames - if set to true, it is assumed first row of csv file contains field names; if set to false map variable is used
   • map - defines fieldnames for files where first row doesn't contain fieldnames
   • single_sheet - tells if parsed should be one sheet or all; should contain name of the sheet to be parsed
   • adapter - tells which adapters method should be called on each row
   • method - tells which method of adapter should be called on each row; defaults to saveRow

3. Standard customer and product entity parsers

   For most commonly exchanged entities - customer and product - Magento provides also standard parsers: customer/convert_parser_customer (Mage_Customer_Model_Convert_Parser_Customer) and catalog/convert_parser_product (Mage_Catalog_Model_Convert_Parser_Product). Both inherit from Mage_Eav_Model_Convert_Adapter_Entity.

   Since standard adapter's load() methods calls result with array of solely entities' id values it is required to call parser's unparse method, if we want to get more related data. Both parsers take this arrays and for each entity parse its data variable content, ignore system fields, objects, non-attribute fields and create an associative array from the rest. Additionally product parser add to the array result of parsing product related stock item object, and customer parser - result of parsing shipping and billing addresses and information about newsletter subscription.

   Both entities parsers have deprecated parse() methods, since their function is now mostly done by parser actions with standard adapter methods called within parser's context. Example of product parser definition, parsing only products from selected store:

   <actiontype="catalog/convert_parser_product"method="unparse">
       <varname="store"><![CDATA[1]]></var>
   </action>

4. Mapping values

   DataFlow module provides also a mapper concept - class with map() method that is responsible for mapping processed fields from one to another. The definition of mapper looks like that for example:

   <actiontype="dataflow/convert_mapper_column"method="map">
       <varname="map">
           <mapname="category_ids"><![CDATA[categorie]]></map>
           <mapname="sku"><![CDATA[reference]]></map>
           <mapname="name"><![CDATA[titre]]></map>
           <mapname="description"><![CDATA[description]]></map>
           <mapname="price"><![CDATA[prix]]></map>
           <mapname="special_price"><![CDATA[special_price]]></map>
           <mapname="manufacturer"><![CDATA[marque]]></map>
       </var>
       <varname="_only_specified">true</var>
   </action>

   Again we have action tag with two attributes: type set as mapper class alias and method that is called to do the mapping. Mapper dataflow/convert_mapper_column is a standard mapper you can find in Magento DataFlow module within app/code/core/Dataflow/Model/Mapper/ folder, and its purpose is to map one array into another with changing the name and posibility to limit fields in result. Map's tag attribute name tells which field name should be replaced in new array by field named like the content of map's tag. If named field doesn't exist in source array, value for target's array field is set to null. Variable _only_specified tells if only fields specified in map definition should be in the resulting array.

This article would be the one that close standard features of DataFlow module and basics of its usage.