SDMX-HD Data Export -- Zanzibar
Background Information
iHRIS and DHIS are established systems in Zanzibar. Each has independently maintained a list of facilities and districts on which data should be exchanged. In addition iHRIS contains cadre, job and gender lists on which health worker data is maintained. In the case of Zanzibar, it was decided that exporting data at the level of job (approximately 200) provided the needed granularity for the expected data use. In other cases, it may make sense to export on cadre -- in the case of Botswana there are 3500 jobs which is perhaps too granular for meaningful use for planners and decision/policy makers.
Expected Data Use Scenarios
- Computer # of deliveries per healthworker in job that delivers maternal child health services
Where's the code
There is a Launchpad team and project. The direct link to the source code is here.
There are four main directories here and it is intended that these tools can be branch and readily adapted to other country/ministries. There is a script "runme.php" which processes these four directories according to the following logic.
- inputs: this contains a series of Linking the Data files which are the data lists. As an intermediary step, runme.php produces a file #lists.xml which converts the .csv into a simple xml file for further XSLT processing.
- transforms: The files here are used to generate the DSD, the xsd's and what other xml based files needed by the various systems.
- transforms_dsd. This directory contains the XSL which will operate directly on the DSD.
- outputs -- this is where are the results are. (Everything under here is bzr ignored)
Note, runme.php of this is setup to to work without any assumptions about about which systems are involved. It can happily handle openMRS, iHRIS and DHIS all reporting on similar but not necessarily identical data.
Linking the Data
lists.xml
iHRIS
All the transforms and setup files are maintained in transforms/iHRIS. There are three things to be done:
- Make the SDMX-HD codelists available as lists in iHRIS
- Link existing list members in iHRIS to the SDMX-HD code lists
- Produce the export report.
Make the SDMX-HD Code Lists Available
Warning: The linkage between people and facilities in the Zanzibar customizations is different than the usual. Usually it goes:
person -> person_position -> position -> facility
In Zanzibar, due to their business requirements, it goes:
person -> person_position -> position -> facility_department -> facility
Issues to Address
- Unlike in the proof-of-concept for Sierra Leone, where we provided the DSD, the DSD was generated a DSD off of data coming from iHRIS and DHIS. This presented some challenges on our side but all of which can be worked around and improved upon. All data lists are in the inputs sub-directory.
- A dxf import file is created by transforming the DSD to import the "jobs" dataelements into dhis - doctor, nurse etc makingsure they all were part of an iHRIS-Staff data element group. It makes sense that the iHRIS system would have the authoritative list of these.
- Rationalizing the orgunits is really important and potentially quite difficult with a large number of them. We cannot risk overwriting or corrupting our dhis orgunit hierarchy so these must be agreed upon first. There are a few possibilities here:
- Ideally the codelist for facilities should probably be maintained by a 3rd system or one or other systems deemed authoritative.
- In future implementations, we can have dhis act as the authoritative reference - ie. start the process by exporting dhis orgunits and compare with what is in iHRIS. Fix iHRIS and/or DHIS to
make sure these are matching.
- iHRIS does not care about the org units as represented in iHRIS and reports out only on the most granular level in common with DHIS.
- iHRIS can already maintain distinct hierarchical relationships among the same data. For example, we do so with the geographical data with the Christian Social Services Commission as they need to organize by both under the administrative groupings as well as the diocese. If needed/useful we can readily import the DHIS hierarchies into iHRIS.
- In Zanzibar we agreed to report job dataelement disaggregated by gender (Male, Female or Unknown). For the moment the sdmx dsd is using the DHIS codelist values for these which shouldn't strictly have to be the case but it was easier for me this way for now. It's not really critical, but it should be improved in that involves some manual fiddling at the moment.
- Currently we have a constraint on the naming of the keyfamily used in the DSD. This is historical. We name it things like KF_345 where 345 is a categorycombo in dhis. That's fairly ugly and also should and will be improved, but it is not a showstopper. Ideally it would be something like KF_HW_BY_FAC_JOB_GEN or KF_HW_BY_FAC_CADRE_GEN. In any case something a bit more descriptive.
- For historical reasons, we are only currently importing monthly periods. We should generalize that to support other sdmx period types - such as quarterly.
The question of authority for codelists is most important. In the absence of an authoritative 3rd party, the DSD (structural metadata definition) must be created with some peer-to-peer collaboration between two or perhaps more systems. In our case iHRIS provided the jobs dataelements and we provided the orgunits and the gender disaggregation codes. We obviously want to reduce that as far as possible, both to make scenarios more easily replicable and to better enjoy the advantages of having a standard. But we are on the way.
Provided we don't have major headaches with incompatible facility lists (Kenya for example has many more than Zanzibar - 7877) we will get the data exchange working and hopefully improve a bit of process along the way.