Kenya Master Facility List: Difference between revisions

From IHRIS Wiki
Line 14: Line 14:
===File Based Strategy Using XML Form Storage===   
===File Based Strategy Using XML Form Storage===   
We have an XML file containing the external data for a form and we want to read the data directly for this form from an XML file without doing any transforms.  We need to define XPATH expressions to access the data.  Although this method is simplest strategy, your ability to transform the source data is very limited.  This is a stragey used for loading OrganisationUnits from DHIS2.
We have an XML file containing the external data for a form and we want to read the data directly for this form from an XML file without doing any transforms.  We need to define XPATH expressions to access the data.  Although this method is simplest strategy, your ability to transform the source data is very limited.  This is a stragey used for loading OrganisationUnits from DHIS2.
===Service Based Strategy Using XMLDB Form Storage==
===Service Based Strategy Using XMLDB Form Storage===
There is an external system (or service) with an API to access the data for a form and there is a reliable internet connection between iHRIS and the external system.  In this case, we use the new XML Database Storage mechanism that is version 4.2  of iHRIS.  This is the strategy used for the OpenHIE Health Worker Registry
There is an external system (or service) with an API to access the data for a form and there is a reliable internet connection between iHRIS and the external system.  In this case, we use the new XML Database Storage mechanism that is version 4.2  of iHRIS.  This is the strategy used for the OpenHIE Health Worker Registry
==Kenya Master Facility List==
==Kenya Master Facility List==
The Kenya MFL Is available from http://www.ehealth.or.ke/facilities
The Kenya MFL Is available from http://www.ehealth.or.ke/facilities

Revision as of 11:24, 12 May 2014

These is a capture of a [interactive session] on linking iHRIS to a Master Facility List

Overview

We are going to look at importing XML based data into iHRIS from an external system. As this data is coming from an external system, we expect that the data will change over time. Therefore, we want to be able to continually refresh the imported data. There are three main strategies for iHRIS to easily import such XML data. The first two are "file" based and the third is "service" based. In the first and third of these strategies, we can use XSL transforms to take our source data and transform it into the expected format needed by iHRIS. The second method only uses XPATH expressions to navigate the data in the source XML document. In this tutorial we will use the first strategy to transform the Kenya Master Facility List into the "mfl_facility" form in iHRIS

We will proceed in a few steps.

  • The first step is to look at the MFL and see what data we want to extract from it.
  • The second step is to map the MFL Districts in the iHRIS county form
  • The third step is to generate iHRIS facility data under /I2CE/formsData/forms/facility using the MFL using the faiclity names and codes
  • The fourth step is to link in the MFL Districts / iHRIS county into the /I2CE/formsData/forms/facility

File Based Strategy To Magic Data

We have an XML file containing the external data for a form and we want to store the data for this form in magic data. We will want to transform the data into an module defining the magic data living under /I2CE/formsData/forms/$form.

File Based Strategy Using XML Form Storage

We have an XML file containing the external data for a form and we want to read the data directly for this form from an XML file without doing any transforms. We need to define XPATH expressions to access the data. Although this method is simplest strategy, your ability to transform the source data is very limited. This is a stragey used for loading OrganisationUnits from DHIS2.

Service Based Strategy Using XMLDB Form Storage

There is an external system (or service) with an API to access the data for a form and there is a reliable internet connection between iHRIS and the external system. In this case, we use the new XML Database Storage mechanism that is version 4.2 of iHRIS. This is the strategy used for the OpenHIE Health Worker Registry

Kenya Master Facility List

The Kenya MFL Is available from http://www.ehealth.or.ke/facilities

There is a web-api that will let you download the facility list as an XML document. You need a username and password to access the API. There is an example below.

Example MFL

Here is an example facility record returned from the MFL API:

 https://gist.github.com/litlfred/9765245

Here is an example for one facility. <source lang='xml'> <DocumentElement>

<Facility>
  <FAC_Code>14753</FAC_Code>
  <FAC_Name>Kapsara District Hospital</FAC_Name>
  <FAC_Type>District Hospital</FAC_Type>
  <FAC_Owner>Ministry of Health</FAC_Owner>
  <FAC_County>Trans Nzoia</FAC_County>
  <Fac_District>Trans Nzoia East</Fac_District>
  <FAC_Division>Kaplanai</FAC_Division>
  <FAC_Location>Makutano</FAC_Location>
  <FAC_SubLocation>Kapsara</FAC_SubLocation>
  <FAC_Constitiuency>NOT IN LIST</FAC_Constitiuency>
  <FAC_Nearesttown>Kitale</FAC_Nearesttown>
  <FAC_KEPHlevel>Level 4</FAC_KEPHlevel>
  <FAC_PlotNo/>
  <FAC_Latitiude>1.07219</FAC_Latitiude>
  <FAC_Longitude>35.1511</FAC_Longitude>
  <FAC_GEOSource>KEMRI GIS MERGE</FAC_GEOSource>
  <FAC_GEOMethod>2</FAC_GEOMethod>
  <FAC_GEOName>Calc from proximity to school, village, markets</FAC_GEOName>
  <FAC_GeoDate>2009-07-14T00:00:00-04:00</FAC_GeoDate>
  <FAC_OfficalLandline/>
  <FAC_OfficalFAX>020-2394909020-2394909020-2394909020-2394909020-2394909020-2394909020-2394909020-2394909020-2394909020-2394909</FAC_OfficalFAX>
  <FAC_OfficialMobile/>
  <FAC_OfficialEmail/>
  <FAC_AddressBOX>P.O. Box 2234</FAC_AddressBOX>
  <FAC_AddressTown>Kitale</FAC_AddressTown>
  <FAC_AddressPostCode>30200</FAC_AddressPostCode>
  <FAC_BedsNo>22</FAC_BedsNo>
  <FAC_CotsNo>0</FAC_CotsNo>
  <FAC_Open24Hours>0</FAC_Open24Hours>
  <FAC_OpenWeekends>0</FAC_OpenWeekends>
  <FAC_OfficialName>Kapsara District Hostpital</FAC_OfficialName>
  <FAC_ActiveStatus>Operational</FAC_ActiveStatus>
</Facility>

</DocumentElement> </source>

Identifying Needed Data

From the Kenya MFL, we will want to import a few of the data fields into iHRIS. We could of course import all of them, if we wanted to, but let us focus on the key data fields for today. These key data fields should include:

  • FAC_Code -- used to link the iHRIS Facility lists to other systems. For example, when we report facility based data, such as number of healthworkers in a facility, we will want to use the facility code.
  • FAC_Name or FAC_OfficialName: We should have at least one of these, if not both. We will start with using FAC_Name because that one is always populated.
  • FAC_District: Normally, I would such including this field if there was a code associated with it. However, it appears to be free-text only. Because of this, it will be difficult to link the districts in iHRIS to the FAC_District field.
  • FAC_County: The same applies for FAC_County and other geographic information. However, we need this data for adding in new facilities with their county

Exporting Current Counties in iHRIS

We need to be able to match FAC_County with the current counties in iHRIS. In order to do this, we will need to export them using the Magic Data Browser.

NOTE: If the source code for the Kenya customizations were on Launchpad, we could simply look at that. To get to the Magic Data Browser, click on:

  • Configure System
  • Browse Magic Data

The counties are stored in magic data under /I2CE/formsData/forms/county. To get there we now click on:

  • I2CE
  • formsData
  • forms
  • county

Now you should see an export button. If you click on this, a window will pop-up. We will need to fill in the module name and display name for this module.

  • module name = data-kenya-county
  • module diplay name = Data for Counties in Kenya

Once you have entered these fields you can click the "Export" button and you should be able to download an XML file

These counties have been posted here:

 https://gist.github.com/litlfred/9782238

Structure of the Exported county form (which is actually called a District)

Here is a snippet of the exported counties. There are two main sections to these module XML files The first section is the "metadata" section. This is what is used to define the module. It's key element is the "version" which tells us the version of the module and is used to track the changes to the data/module over time.

The second section starts with the <source lang='xml'>

<configurationGroup name='County' path='/I2CE/formsData/forms/county'>

</source> This is all of the data that is defined for this module. The @path attribute says that what is inside this configurationGroup yes In the example below, we have two configurationGroup elements under <source lang='xml'>

<configurationGroup name='County' path='/I2CE/formsData/forms/county'>

</source> The first is <configurationGroup name='1'/&gt. The second is <configurationGroup name='10'/>

These two <configurationGroup/> elements define the data for county|1 and county|10 respectively. Let's take a look at the <configuraitonGroup name='1'/> in a bit more details. It has two sub-elements:

  • <configurationGroup name='fields'/> which contains all of the field data for county|1
  • <configurationGroup name='last_modified'/> which tells us when this form was last modified.

Looking under <configurationGroup name='fields'> we have the following <configrationGroup name='$field'> where $field is:

  • district: this is the district that the county is in. In this case the value is district|1
  • i2ce_hidden: this is an internal field used to mark whether or not this county should be displayed in the drop-downs. In this case, the value=0 so we do not want to hide it, we want to show it
  • name: this is the name of the county. In this case "Bahari"
  • remap: this is an internal field. it is used to remap county data.

So how do we know what district|1 is? We need to look at the data module export for the distirct form (which is actually a County!) The exported data module for districts are here: https://gist.github.com/litlfred/9782636 Taking a look we see that district|1 is the County named Kilifi <source lang='xml'> <I2CEConfiguration name="County">

 <metadata>
   <displayName>County</displayName>
   <version>1.0.2014.03.26</version>
 </metadata>
 <configurationGroup name="County" path="//I2CE/formsData/forms/county">
   <displayName>Export</displayName>
   <version>1.0.2014.03.26</version>
   <configurationGroup name="1">
     <displayName>1</displayName>
     <configuration name="who">
       <displayName>Who</displayName>
       <value></value>
     </configuration>
     <configurationGroup name="fields">
       <displayName>Fields</displayName>
       <configuration name="district">
         <displayName>District</displayName>
         <value>district|1</value>
       </configuration>
       <configuration name="i2ce_hidden">
         <displayName>I2ce Hidden</displayName>
         <value>0</value>
       </configuration>
       <configuration name="name">
         <displayName>Name</displayName>
         <value>Bahari</value>
       </configuration>
       <configuration name="remap">
         <displayName>Remap</displayName>
         <value></value>
       </configuration>
     </configurationGroup>
     <configuration name="last_modified">
       <displayName>Last Modified</displayName>
       <value>2013-10-15 11:28:29</value>
     </configuration>
   </configurationGroup>
   <configurationGroup name="10">
     <displayName>10</displayName>
     <configuration name="who">
       <displayName>Who</displayName>
       <value></value>
     </configuration>
     <configuration name="last_modified">
       <displayName>Last Modified</displayName>
       <value>2013-10-15 11:28:31</value>
     </configuration>
     <configurationGroup name="fields">
       <displayName>Fields</displayName>
       <configuration name="district">
         <displayName>District</displayName>
         <value>district|7</value>
       </configuration>
       <configuration name="i2ce_hidden">
         <displayName>I2ce Hidden</displayName>
         <value>0</value>
       </configuration>
       <configuration name="name">
         <displayName>Name</displayName>
         <value>Bungoma South</value>
       </configuration>
       <configuration name="remap">
         <displayName>Remap</displayName>
         <value></value>
       </configuration>
     </configurationGroup>
   </configurationGroup>

</source>

Linking the Kenya MFL to the iHRIS Data

We now have seen an example of the MFL and the iHRIS data for the county and district forms (called Districts and County, confusingly). Now we have a slight problem. In the MFL we have a County like <source lang='xml'>

<FAC_County>Trans Nzoia</FAC_County>

</source>

But in iHRIS we would want to translate that into something like district|12

Whenever we try to match based on text fields, we almost always end up with problems. For example, the corresponding County in iHRIS could have been entered as:

  • TRANS NZOIA
  • Trans-Nzoia
  • TRANS-NZOIA

or event something completely different. This is especially true when we get into abbreviations. For example you could have something like "Eastern Nzoia" in the MFL, but in iHRIS you could have:

  • Eastern Nzoia (and you are lucky)
  • East Nzoia
  • E. Nzoia
  • E NZOIA
  • etc...

Ideally, the District and Counties in the Kenya MFL would use codes, and then we could use the same codes in iHRIS to match things. As it is, we have a problem, for which there are no easy answers....The most realiable thing to do is create a lookup table.

Let's take a look at what is in the iHRIS Facility data. So that we can see if we are linking faciltiies to the county form or district form.

Creating a Lookup Table as an XML file

Let us assume that we are creating a look up table for Districts. We want to pair the MFL District name to the correct county form (remember the confusion!) Because we want to use XSL for transforming the MFL into an iHRIS data module, it is easiest if we create this lookup table as an XML file. Note: XSL is a way of transforming XML documents. The actual structure of the lookup table as XML doesn't really matter. So I will suggest something simple now. I will make it easy to extract from the MFL which uses the FAC_District So let's start by extracting each of the districts from the MFL. As an example we would get: <source lang='xml'> <lookupDistrict>

 <FAC_Distrct>Trans Nzoia East</FAC_District>
 <FAC_Distrct>Trans Nzoia West</FAC_District>

</lookupDistrict> </source> Now we want to add some data to this XML file to link to the correct iHRIS county form. This simplest way is to add an attribute to each FAC_District: So let's start by extracting each of the districts from the MFL. As an example we would get: <source lang='xml'> <lookupDisitrct>

 <FAC_Distrct ihris_county="county|196" >Trans Nzoia East</FAC_District> 

</lookupDistrict> </source> Someone now needs to finish creating this XML file for all of the MFL Districts. It will take a bit of time, but I think there are only about 260 of them so it is only one tedious afternoon. Luckily there are not thousands of them....like there are with facilities. But luckily the facilites have codes!

XPATH

Before we get started with XSLT, we will need to know a bit about XPATH. XPATH is the XML way of defining a path in the XML to pull out a node or an element. There is a tutorial here: http://www.w3schools.com/XPath/ but we will just look at a few simple expressions for what we need. A few examples:

  • To get all of the FAC_District nodes in the MFL we can use the expression //Fac_District
 at this point you should copy the XML document in https://gist.githubusercontent.com/litlfred/9765245/raw/a622cf2f1af1a7a7a8bdb3eb1b25a2a0e7caf900/mfl_sample.xml 

into the online XPATH evaluator http://www.online-toolz.com/tools/xpath-editor.php and enter in the XPATH expression //Fac_District The result is..... (The ----- are added by the online evaluator) <source lang='xml'>

 <Fac_District>Trans Nzoia East</Fac_District>
 <Fac_District>Trans Nzoia East</Fac_District>
 <Fac_District>Trans Nzoia East</Fac_District>
 <Fac_District>Trans Nzoia East</Fac_District>

</source>

  • Now what if you only wanted the text content, and not the <Fac_District/%gt; tags? You can do that with //Fac_District/text(). The result is:

<source lang='bash'>

 Trans Nzoia East
 Trans Nzoia East
 Trans Nzoia East
 Trans  Nzoia East

</source>

  • What happens with /Fac_District Is there any result? There is no result. The "//" means to search at any depth in the XML document. When you only have a single slash,
 you are saying that you are looking for Fac_District under the root (top-level) element
  • If you do /DocumentElement you will get the whole document
  • Now let's take something a bit more complicated.
Let's look at the export iHRIS county form data:https://gist.githubusercontent.com/litlfred/9782238/raw/bbb4107530a583506dd569c4a95de876e3209007/ihris_kenya_counties.xml
and copy that into the online editor.
Now enter the XPath:  //configurationGroup[@name='fields']/configuration[@name='name']/value
This will print out all the names of the counties.  Let's take this apart piece by piece:
**//configuratioGroup says that we are looking for a <configurationGroup/> element at any level
**//configuratioGroup[@name='fields'] says that we are looking for a <configuartionGroup/> at any level with the attribute @name having the value "fields"
**//configuratioGroup[@name='fields']/configuration[@name='name']  says we are looking for a <configuartionGroup/%gt; at any level with the attribute @name having the value "fields" 
  and which has a sub-element  <configuration/&gtl  with attribute @name with value name
**finally    //configurationGroup[@name='fields']/configuration[@name='name']/value says that we want the value element under the <configuration name='name'/> element


http://www.online-toolz.com/tools/xpath-editor.php

XSLT

An XSL transformation (or XSLT) is a way to change one XML document into another XML document. In our case our input/source document is the Kenya Master Facility List. Our output/target document will be an iHRIS module which contains the facility data. It will look something like the "County" data starting on line 175 above.

XSLT Wrapper

There are lots of options when working with XSL. However, there is a good template that you should start from: <source lang='xml'> <?xml version="1.0" encoding="UTF-8"?> <xsl:stylesheet version="1.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform">

 <xsl:output method="xml" version="1.0" encoding="UTF-8" indent="yes"/>
 <xsl:template match="/">
 </xsl:template>

</xsl:stylesheet> </source>

  • In the above, we on line 344 , the declaration that this is an XML document (an XSL document is an XML document)
  • On line 345 (and closing on line 350), we are declaring that this specific XML document is a XSL stylesheet. XSL = eXtensible Stylesheet Language
  • Line 346, tells us that our output is also an XML docuemnt.
  • Line 347 (and closing on line 349) tells us that we are going to apply our transformation to the entire input document ( the Kenya MFL)

This "wrapper" is your starting point for transforming the MFL. It doesn't really do anything interesting yet. We need to change the " " to make it do something interesting.

XSLT Looping/For Each

Our source document (the Kenya MFL) has a list of facilities. See a sample here:

 https://gist.github.com/litlfred/9765245

We want to loop through each of these facilities to produce our output iHRIS XML document. To do this you use the <xsl:for-each select="XPATH EXPRESSION"/> like so: <source lang='xml'> <?xml version="1.0" encoding="UTF-8"?> <xsl:stylesheet version="1.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform">

 <xsl:output method="xml" version="1.0" encoding="UTF-8" indent="yes"/>
 <xsl:template match="/">
     <xsl:for-each  select="//FAC_Name">    
              <name><xsl:value-of select="text()"/></name>
      </xsl:for-each>
 </xsl:template>

</xsl:stylesheet> </source>

xsltproc

We should save the sample Kenya MFL to a file, for example mfl_example.xml We should also save our XSL to a file. Let's save the above to mfl_names.xsl We can do the transformation on the command line like: <source lang='bash'>

  xsltproc <STYLESHEET> <SOURCEXMLFILE>

</source> In our example it would be <source lang='bash'>

 xsltproc mfl_names.xsl mfl_example.xml

</source> To run xsltproc, you may need to do: <source lang='bash'>

 sudo apt-get install xsltproc

</source> first. The output is: <source lang='xml'> <?xml version="1.0" encoding="UTF-8"?> <name>Kapsara District Hospital</name><name>Kapsara Pharmacy</name><name>Kapsara PHC</name><name>Kapsara North PHC</name> There is a problem with this output. It is not a valid XML document! The reason is that there is no "top-level" node, or "document" element. But this is because we did not include that in our XSL. Let's put a <names/> element to enclose all of the <name>Kaspara District Hospital</name><name>.....</name> elements <?xml version="1.0" encoding="UTF-8"?> <xsl:stylesheet version="1.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform">

 <xsl:output method="xml" version="1.0" encoding="UTF-8" indent="yes"/>
 <xsl:template match="/">
     <names>
         <xsl:for-each  select="//FAC_Name">    
              <name><xsl:value-of select="text()"/></name>
         </xsl:for-each>
     </names>
 </xsl:template>

</xsl:stylesheet> </source> What this says is that we create a <names> element, loop through each of the //FAC_Name and then create a <name/> for each. Output: <source lang='xml'> <?xml version="1.0" encoding="UTF-8"?> <names>

 <name>Kapsara District Hospital</name>
 <name>Kapsara Pharmacy</name>
 <name>Kapsara PHC</name>
 <name>Kapsara North PHC</name>

</names> </source>

Example: Get The List Of Facility Codes into an iRHIS Data Module

We are going to use the MFL codes to procuce the ID of the iHRIS Facility. If the MFL Code is, for example "123", then we will make the corresponding iHRIS ID "mfl_123". THis way we can recognize when the facility came from iHRIS "facility|1" or was imported from the MFL as "facility|mfl_123" We also want to go ahead and start creating the module for iHRIS sorta like on line 175 above. So let us start with the XSL wrapper/template and add in the some of the iHRIS specific stuff. Instead of looping through each of the <FAC_Name/> elements. This time let us loop through the <Facility/> elements as we want both the code and name... <source lang='xml'> <?xml version="1.0" encoding="UTF-8"?> <xsl:stylesheet version="1.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform">

 <xsl:output method="xml" version="1.0" encoding="UTF-8" indent="yes"/>
 <xsl:template match="/">
   <I2CEConfiguration name="mfl_data">  
     <metadata>
      <displayName>The Kenya Master Facility List Data</displayName>  
      <version>4.1.9.2014.04.07.1</version>  
    </metadata>
    <configurationGroup name="County" path="//I2CE/formsData/forms/facility">  
      <version>4.1.9.2014.04.07.1</version>
      <xsl:for-each select="//Facility">
           <configurationGroup >    
               <xsl:attribute name='name'>mfl_<xsl:value-of select="FAC_Code/text()"/></xsl:attribute>  
               <configurationGroup name='fields'>
                    <configuration name='name' locale='en_US'>
                       <value><xsl:value-of select="FAC_Name/text()"/></value>  
                    </configuration>                     
                    <configuration name='code' >
                       <value><xsl:value-of select="FAC_Code/text()"/></value>   
                    </configuration>  
                    <configuraiton name='location'>
                       <value><xsl:value-of select="FAC_District/text()"/></value>
                    </configuration>                     
               </configurationGroup>
           </configurationGroup>
      </xsl:for-each>
       </configurationGroup>
     </I2CEConfiguration>
 </xsl:template>

</xsl:stylesheet> </source> If we execute the above XSL against our sample MLF, we get the following output: <source lang='xml'> <?xml version="1.0" encoding="UTF-8"?> <I2CEConfiguration name="mfl_data">

 <metadata>
   <displayName>The Kenya Master Facility List Data</displayName>
   <version>4.1.9.2014.04.07.1</version>
 </metadata>
 <configurationGroup name="County" path="//I2CE/formsData/forms/facility">
   <version>4.1.9.2014.04.07.1</version>
   <configurationGroup name="mfl_14753">
     <configurationGroup name="fields">
       <configuration name="name" locale="en_US">
         <value>Kapsara District Hospital</value>
       </configuration>
       <configuration name="code" >
         <value>14753</value>
       </configuration>
       <configuration name="location" >
         <value>Trans Nzoia East</value>
       </configuration>
     </configurationGroup>
   </configurationGroup>
   <configurationGroup name="mfl_14754">
     <configurationGroup name="fields">
       <configuration name="name" locale="en_US">
         <value>Kapsara Pharmacy</value>
       </configuration>
       <configuration name="code" >
         <value>14754</value>
       </configuration>
       <configuration name="location" >
         <value>Trans Nzoia East</value>
       </configuration>
     </configurationGroup>
   </configurationGroup>
   <configurationGroup name="mfl_14755">
     <configurationGroup name="fields">
       <configuration name="name" locale="en_US">
         <value>Kapsara PHC</value>
       </configuration>
       <configuration name="code" >
         <value>14755</value>
       </configuration>
       <configuration name="location" >
         <value>Trans Nzoia East</value>
       </configuration>
     </configurationGroup>
   </configurationGroup>
   <configurationGroup name="mfl_14757">
     <configurationGroup name="fields">
       <configuration name="name" locale="en_US">
         <value>Kapsara North PHC</value>
       </configuration>
       <configuration name="code" >
         <value>14757</value>
       </configuration>
       <configuration name="location" >
         <value>Trans Nzoia East</value>
       </configuration>
     </configurationGroup>
   </configurationGroup>
 </configurationGroup>

</I2CEConfiguration> </source>

Adding Files As A Module To Launchpad

Before continuing, we will add the files above to our bzr source code repository. We will want to create a new module to contain the master facility list, the xsl as well the data module. All modules go under the directory "modules". I suggest that we use the following directory layout

  • modules/MFL the directory containing all the files for the MFL
  • modules/MFL/MFL.xml this will be the output of the transformation of the MFL by XSL
  • modules/sources/MFL_source.xml This is the source MFL downloaded from the MFL website
  • modules/sources/MFL_to_data.xsl This is the transform that we are using.

To do this we do: <source lang='bash'>

cd /path/to/kenya/customizations
mkdir -p modules/MFL/sources

</source> this will create all the direcotries that we need. Now let's create the MFL_to_data.xsl by: <source lang='bash'>

nano modules/sources/MFL_to_data.xsl

</source> copy the contents of lines 435-478 into nano, save and exit. You should also save a copy of the MFL downloaded from their website into:

 modules/source/MFL_source.xml
 

Once you have added the files above, you can add them to bzr with: <source lang='bash'>

cd /path/to/kenya/customizations
bzr add modules/MFL

</source> Before you commit, you should update. You do: <source lang='bash'>

 bzr update 
 bzr commit -m "added source files for MFL"

</source>

The next thing we would want to do is run the XSL transform to create the MFL data module for iHRIS: <source lang='bash'>

   cd /path/to/kenya/customizations/modules/MFL/sources
   xsltproc MFL_to_data.xsl MFL_source.xml > ../MFL.xml

</source>

If there are now errors, we should now have the modules/MFL/MFL.xml file. We can check this with: <source lang='bash'>

    cd /path/to/kenya/customizations/modules/MFL
    bzr status

</source> You should see the output like: <source lang='bash'>

 unknown:
 modules/MFL/MFL.xml

</source> which means that we have not yet added this file. So you need to do: <source lang='bash'>

  bzr add MFL.xml  

</source> and (after testing that the module is avaiable and loads) we are now ready to commit: <source lang='bash'>

   bzr commit MFL.xml -m "Added MFL data module"

</source>


Comitted Code

The current MFL data module in iHRIS Manage Kenya can be found here:

 http://bazaar.launchpad.net/~ihris+kenya/ihris-kenya/ihris-manage-4-1/files/head:/modules/MFL/

under the file MFL.xml Currently there are no locations assigned to the faiclity. The file MFL.xml was generated with: <source lang='bash'>

 kenya/modules/MFL/sources$ xsltproc --stringparam unixTime "`date +%s`" MFL_to_data.xsl MFL_source.xml > ../MFL.xml

</source> (see the section of XSLT and external variables to see how we got the unix timestamp as part of the module verison)

=Lookup Location With A Lookup Table

On line 270-297 above, we talked about creating a lookup table for districts and counties. A lookup table was created to change Fac_District to an iHRIS county form. The table was commited in the MFL sources directory here:

 http://bazaar.launchpad.net/~ihris+kenya/ihris-kenya/ihris-manage-4-1/view/6/modules/MFL/sources/lookuptable_districts.xml

The current (at revision 6) transform of the MFL to an iHRIS data module for facilties is here:

  http://bazaar.launchpad.net/~ihris+kenya/ihris-kenya/ihris-manage-4-1/view/head:/modules/MFL/sources/MFL_to_data.xsl

on lines 30-37 we have: <source lang='xml'>

                   <configuration name='location'>
                       <value><xsl:value-of select="Fac_District/text()"/></value>
                    </configuration>   

</source >

which is just putting in the Fac_District/text().   We want to change this so that is uses the lookup table to change Fac_District/text() into somehting like county|123

What we will want to do is to assign the values like county|123 to a variable.  So first we need to review how to declare variables in XSL.  The syntax is:

<source lang='xml'>

  <xsl:variable name='blah' select='XPATH EXPRESSION'/>

</source>

Once we have declared the variable "blah" we can use it for example like:

<source lang='xml'>

  <xsl:value-of select="$blah"/>      

</source> Now the trick to the lookup table is to write an xpath expression to assign a variable that uses an external document. for this, we can use the document() function in xpath:

 http://www.w3schools.com/xsl/func_document.asp

Suppose we want a variable named $ihris_county from the lookup table. Then we can do: <source lang='xml'>

 <xsl:variable name="ihris_country" select="document('lookuptable_districts.xml)//Fac_District[text()  = Fac_District/text()]/>

</source> This is perhaps a bit convoluted, but would work (I think). So let's do something that is a litle bit clearer. First, let's assign a variable with the value of the district name: <source lang='xml'>

  <xsl:variable name='district' select="Fac_District/text()"/>   
  <xsl:variable name='ihris_county' select="document('lookuptable_districts.xml')//Fac_District[ text() = $district]/@ihris_county"/>

</source>

The XPath expression on line 630 says to us:  open up the lookuptable_districts.xml document.  Lookup any Fac_District element whose text() value matches the $district variable
    

Assuming that there are no bugs ...and that is no gaurantee ;-) .... we should now have a valid country|123 in the $ihris_county variable. Pulling all of this together, we want to replace lines 30-37 of our MFL_to_data.xsl with the following: <source lang='xml'>

                  <configuration name='location'>
                      <xsl:variable name='district'  select="Fac_District/text()"/>   
     <xsl:variable name='ihris_county'  select="document('lookuptable_districts.xml')//Fac_District[ text() =  $district]/@ihris_county"/>  
                       <value><xsl:value-of select="$ihris_county"/></value>
                    </configuration>  

<source> Now we rerun the xsltproc command: <source lang='bash'>

 cd modules/MFL/sources
 xsltproc --stringparam unixTime "`date +%s`" MFL_to_data.xsl MFL_source.xml > ../MFL.xml

<source> we check that there are no errors. We can also check the changes since the last commit with "bzr diff" to see output like: <source lang='bash'>

 === modified file 'modules/MFL/MFL.xml'

--- modules/MFL/MFL.xml 2014-04-23 13:20:19 +0000 +++ modules/MFL/MFL.xml 2014-04-23 14:05:54 +0000 @@ -2,10 +2,10 @@

<I2CEConfiguration name="mfl_data">
  <metadata>
    <displayName>The Kenya Master Facility List Data</displayName>

- <version>4.1.9.1398259149</version> + <version>4.1.9.1398261951</version>

  </metadata>
  <configurationGroup name="County" path="//I2CE/formsData/forms/facility">

- <version>4.1.9.1398259149</version> + <version>4.1.9.1398261951</version>

    <configurationGroup name="mfl_16548">
      <configurationGroup name="fields">
        <configuration name="name" locale="en_US">

@@ -15,7 +15,7 @@

          <value>16548</value>
        </configuration>
        <configuration name="location">

- <value>Msambweni</value> + <value>county|136</value>

        </configuration>
      </configurationGroup>
    </configurationGroup>

</source> This output tells us that the file MFL.xml (our facility data module file) has changed. You can see that the verison number has increased. You can also see that instead of Msambweni we have country|136 in the location field So to me, this all looks OK and we are ready to commit: <source lang='bash'>

 bzr commit -m "updated the XSL transform for the MFL to use the lookuptable to change the District name to an ihris county form.  Reran xsltproc"

</source>

And we are done!

XSLT and External Variables

You can get the current unix timestamp (seconds since the Unix Epoch, Jan 1, 1970) on the command line: <source lang='bash'>

 date +%s

</source> To pass this into xsltproc you can use the following: <source lang='bash'>

  xsltproc --stringparam unixTime "`date +%s`" <STYLESHEET> <SOURCEXMLFILE>

</source> This is useful for updating the module's version according to a time stamp.

Linking Old Facilities to the MFL Facilities

Setting Remapping Data

The facility form is a list. Every list form has the field "remap" that let's you point an old list member(or old facility) to a new list member(or new facility). For example, if there is an old facility:

    • id = facility|123
    • name = Central Hospital
    • location = district|123

and a new facility

    • id = facility|mfl_456
    • name = Central Hospital
    • location = district|123

We can set the old Central Hospital, with id = 123 to the new Central Hospital with id=mfl_456 So what we want is that the old facility

    • id = facility|123
    • name = Central Hospital
    • location = district|123
    • remap = facility|mfl_456

Updating The Linking Data

Once the remapping data has been set you can update all of the forms that point to the old facility to point to the new facility. This can be down by going to:

http://localhost/manage/index.php/view/auto_list?type=facility

See also http://wiki.ihris.org/wiki/Remap_Data_Fields

Bulk Remapping

We can (as described in the wiki link above) use the web-interface under the Auto List page to set the remapping data. This is fine if we only have 5-10 or so facilities to do. However, we now have a lot of facilities that we want to set the remapping data for. To do this, we will want to write a script to try and match the old faciltiies to the new facilities from the MFL based on the name. There are a couple of ways that we can do this....It is not too important to do this in a "clean" or "nice" way, as it is a one time fix. I would suggest that we do this as a PHP script that we run from the command line.

Script to get all the faciltiies and their names

We will put the following in the pages directory. Once it is in the pages directory as facility_list.php, you can run it as "php facility_list.php" <source lang='php'> <?php $i2ce_site_user_access_init = null; require_once( dirname(__FILE__) . DIRECTORY_SEPARATOR . 'config.values.php'); $local_config = dirname(__FILE__) . DIRECTORY_SEPARATOR . 'local' . DIRECTORY_SEPARATOR . 'config.values.php'; if (file_exists($local_config)) {

   require_once($local_config);

} if(!isset($i2ce_site_i2ce_path) || !is_dir($i2ce_site_i2ce_path)) {

   echo "Please set the \$i2ce_site_i2ce_path in $local_config";
   exit(55);

} require_once ($i2ce_site_i2ce_path . DIRECTORY_SEPARATOR . 'I2CE_config.inc.php'); @I2CE::initializeDSN($i2ce_site_dsn, $i2ce_site_user_access_init, $i2ce_site_module_config); //This is copied from the index.php file. It basically connects to the database and initialize iHRIS. //Let us in this script, loop though all of the existing facilities and display the name field for that facility. //We will be using the I2CE_FormStorage::listFields() method http://wiki.ihris.org/wiki/Class:_I2CE_FormStorage_Mechanism_%284.1.7%29#listFields.28.29 $existing_facilities = I2CE_FormStorage::listFields('facility',array('name')) //the first arguement is the name of the form, the second argument is an array of the fields that we want //the listFields() function will return an array. //the keys of the array are the ids. the values are themselves an array. the keys of the array are the name of the fields we sent in the second argument, the values are what is stored in the DB //example: $existing_facilites = array( '123'=>array('name','Capital City')); //now let's loop through and print out the id and name foreach ($existing_facilities as $id=>$data) {

  if (!is_array($data) || !array_key_exists('name',$data)) {
      //just do some error checking.  if we didn't get an array of data, or the key 'name' wasn't there, let's skip it.
      continue;   
   }    
   echo "Facility with id $id has name ". $data['name']  . "\n";

} </source>

Script to get all the faciltiies and match duplicate names

Now that we know how to list all of the faciltiies, let's create an array of facilities where we look for duplicate names. For example "Capital City" for the ids mfl_456 and 123. We want to match wether it is lower case or upper case, or if there is any variation in special characters like period or comma or spacing. So we will create a "clean" form of the facility name. We copy the code into a file "link_to_mfl.php" in the pages directory. Then we can run "php link_to_mfl.php" from within the pages directory <source lang='xml'> <?php $i2ce_site_user_access_init = null; require_once( dirname(__FILE__) . DIRECTORY_SEPARATOR . 'config.values.php'); $local_config = dirname(__FILE__) . DIRECTORY_SEPARATOR . 'local' . DIRECTORY_SEPARATOR . 'config.values.php'; if (file_exists($local_config)) {

   require_once($local_config);

} if(!isset($i2ce_site_i2ce_path) || !is_dir($i2ce_site_i2ce_path)) {

   echo "Please set the \$i2ce_site_i2ce_path in $local_config";
   exit(55);

} require_once ($i2ce_site_i2ce_path . DIRECTORY_SEPARATOR . 'I2CE_config.inc.php'); @I2CE::initializeDSN($i2ce_site_dsn, $i2ce_site_user_access_init, $i2ce_site_module_config); //This is copied from the index.php file. It basically connects to the database and initialize iHRIS. //Let us in this script, loop though all of the existing facilities and display the name field for that facility. //We will be using the I2CE_FormStorage::listFields() method http://wiki.ihris.org/wiki/Class:_I2CE_FormStorage_Mechanism_%284.1.7%29#listFields.28.29 $existing_facilities = I2CE_FormStorage::listFields('facility',array('name')) $clean_facility_names = array(); foreach ($existing_facilities as $id=>$data) {

  if (!is_array($data) || !array_key_exists('name',$data)) {
      //just do some error checking.  if we didn't get an array of data, or the key 'name' wasn't there, let's skip it.
      continue;   
   }    
   //this is all the same as above, except that we create an array for the clean facilitiy names.
   //clean_facility_names will have keys the "clean facility name" and the value will be an array of matching ids.
   $clean_name = strtoupper($data['name']); //capitalize the facility name
   //now trim out spaces, periods colons and other special characters
   $clean_name = trim($clean_name," .,:-_=+()");  //see http://www.w3schools.com/php/func_string_trim.asp
   //Now we want to add this clean name to our list of clean names, and record the id that goes with it.
   //first check to see if we already saw this clean name before... if not we need to add it to our list.
   if (!array_key_exists($clean_name,$clean_facility_names)) {
       $clean_facility_names[$clean_name] = array();    
   }
   //now we can add it to our list
   $clean_facility_names[$clean_name][] = $id;  //this will append the $id onto the array for the $clean_name
}   

//now we want to look through our list of clean_facility_names to see if there are any duplicates, 
//we should have (in our example) something like:
//$clean_facility_name['CENTRALHOSPITAL'] = array(123,'mfl_456')

$user = new I2CE_User(); ///this will be the user (defaults to admin with id = 0) that will be logged as making the changes to the remap field
$ff = I2CE_FormFactory::instance(); //this is the form factory, which creates the PHP I2CE_Form objects associated to "facility|123" etc.


$unmatched = array();
foreach ($clean_facility_names as $clean_name=>$ids) {
   //if there is only one id, then there wasn't a duplicate..
   //at the very least, we need to skip this one as we don't have any remapping to do.
   //we should probably also print out a warning if the id does not start with 'mfl_'  because that means that we have a facility in iHRIS that is not matching against hte MFL..   We may
   //need to come back and clean these by hand.
   //let's keep track of these unmatched facilties in an array and print them out at the end of the script
   if (count($ids) < 2) {
       $id = current($ids); //this will get the first (and only) member of the $ids array.  http://us2.php.net/current   It is safer than using $ids[0], although in this case if should be fine to use $id = $ids[0]
       if (substr($id,0,4) == 'mfl_') {
            //it is a facility loadded from the MFL.  This is OK   
       }  else {
            //This is a facility that was not loaded from the MFL.  We should add it to our unmatched facility list     
            $unmatched[$id] = $existing_facilties[$id]['name']; 
       }
       continue; //skip if there are not mulitple facilties with the same clean name   
    }
    //if we are here, then we have found at least two faciltiies with the same clean_name.   
    //we want to look through $ids and find the one with an id that starts with "mfl_"
    //two possible errors: either there are no ids starting with mfl_ , in which case we have multiplies facility in iHRIS that do not match the MFL
    //   or there are multiple $id's starting with mfl_  meaning that the MFL has multiple facilties with the same (clean) name.
    // in the first case we should add all the ids to our list of unmatched faciltiies.  in the second case we should complain to the maintainers of the MFL and ask them to fix it.
    //first lets see which ids start with mfl_
    $mfl_ids = array(); //this will hold all the MFL facility ids
    $non_mfl_ids = array(); //this will hold everythign else
    foreach ($ids as $id) {
      if (substr($id,0,4) == 'mfl_') {   
            $mfl_ids[] = $id; 
      } else {
           $non_mfl_ids[] = $id;   
       }
    }
    if (count($mfl_ids) == 0) {
       //badness... no MFL facilties found with this clean name.
       foreach ($ids as $id) {
            $unmatched[$id] = $existing_facilties[$id]['name']; 
        }
       continue;     
    } else if (count($mfl_ids) > 1) {
        //too many MFL faciltiies found
        echo "Bad Master Facilty Clean Name: $clean_name.  Go complaing to MFL maintainer\n";
        continue;
   }   
   //if we made it here, we have found exactly one MFL facility  
   //we can now set the remapping data for the non-mfl facilities.
    //first let's pull out our unique MFL id
    $mfl_id = current($mfl_ids);
    //now we loop through the non MFL faicltiies and set their remapping data
    foreach($non_mfl_ids as $id) {
         //what we need to do is to create an instance of the form with this id as a PHP object
         $facilityObj = $ff->createContainer(array('facility',$id));
         //the form factory will try and create the container.  let's do an error check to be sure...
         if (!$facilityObj instanceof iHRIS_Facility) {
             echo "Could not create a facility with id $id\n";
             continue;    
         }
         //we then populate that form with the existing data in the database
         $facilityObject->populate();
         //now we get the remap field object
         $remapField = $facilityObject->getField('remap');
         //let's make sure it is OK
         if (!$remapField instanceof I2CE_FormField) {
            echo "Could not get remap field!\n";  
            continue;    
         }
         //then we set the remaping field
         $remapField->setValue(array('facility',$mfl_id))
         //then we save it, the $user we created above will be marked as making the changes
         $facilityObject->save($user);
         $facilityObject->cleanup(); ///free up some memory
     }
    
    

} echo "Could not match the following faciltiies against the MFL:\n"; print_r($unmatched);


</source>



Resources