usexmlex - import XML files to Stata by Sergiy Radyakin

usexmlex will import XML data to Stata.

Note that there are multiple ways to store data in XML format, and this program imports only one particular way, which is compatible with the Microsoft Excel XML quieries. Stata itself provides a command xmluse, which imports data stored in a different way.

Description

The usexmlex program will import xml formatted data into Stata. This is an incredibly inefficient format, in which every value is wrapped into a tag of the field, such as (fragment):

<PRESIDENTS>

  <PRESIDENT>
  <lastname>Washington</lastname>
  </PRESIDENT>

  <PRESIDENT>
  <lastname>Adams</lastname>
  </PRESIDENT>

  <PRESIDENT>
  <lastname>Jefferson</lastname>
  </PRESIDENT>

</PRESIDENTS>

This format however is used as an exchange format between various software packages and as a source for quieries to xml data in Microsoft Excel.

This converter makes several assumptions regarding the input data:

• unlike in the regular use command, the extension .xml of the file is not implied, user must specify the file extension;
• flat structure is assumed: no hierarchy or embedded structures;
• entity information (such as PRESIDENTS and PRESIDENT in the above example) is ignored;
• entity tags (PRESIDENT in the above example) must be enveloping all observations; while their exact name is arbitrary, it must be same for all observations, which means no interleaving of objects;
• if data field tags do not satisfy Stata's variable name requirements, an attempt will be made to substitute them with t_TAG, for example, if the original tag was 1984 (number cannot be used as a variable name) then the substitute tag will be t_1984;
• automatic renaming doesn't always succeed, in some cases newly generated variable names can collie with existing, such as if there were another variable called t_1984 already in the data in the above example;
• all fields are assumed to be strings during import, but the program attempts to destring them at the final step.
• entity consistency is not enforced and all attributes are treated as optional. This means that (in the above example) all presidents had the same number and kind of data attribute: lastname. Should e.g. the 4th president have another attribute not encountered before, it will be assigned missing for all entities where it is not specified (this behavior of the program may change in the future).
• there must be at least one entity in the xml data (non-empty dataset).

Installation instructions

Minimum requirement for usexmlex is Stata 13.0, usexmlex is implemented in Mata and should work on any platform, on which Stata itself can run, but the testing was done on MS Windows platform only.

Usexmlex is part of Statistical Software Components (SSC). Individual files are available from this RePEc page. Most users don't need to download individual files and can install with a single command.

To install -usexmlex- type in Stata:

findit usexmlex

Check with SSC regularly to receive further updates:

adoupdate usexmlex

Syntax

The main syntax is trivial:

usexmlex

will show the dialog to select the file name of the file to be imported and any additional parameters.

usexmlex using "datafile.xml"

will import the data from the specified data file. Specify full path if necessary. The word using is mandatory and cannot be omitted from the syntax.

usexmlex about

will display program version and author information.

To load only a subset of data a variables list may be passed in the varloadlist option. For example, to load only two variables id and name from the same file one can write:

usexmlex using "test.xml", varloadlist(id name)

If necessary, an option clear can be added to allow clearing the memory when it contains unsaved changes.

Note that many XML files produced with modern software are stored in a unicode encoding. Stata does not work with unicode, so import such files with caution. For non-latin characters of UTF-8 encoding you will get unrecognizable text in Stata and for other Unicode versions the program may produce unexpected results.

Tutorial and Examples

In the following example we instruct Stata to import an example xml data file

usexmlex using "http://www.radyakin.org/transfer/usexmlex/testdata/test.xml"

The last observation contains a missing value for the name variable. Review the xml file to understand how missing values are stored.

Author and support

In case you are experiencing a problem importing a valid dataset, and you think the error is mine, kindly let me know.

usexmlex was written by Sergiy Radyakin. To contact the author send email to
sradyakin/at/worldbank.org.

Or write your questions to Statalist with tag usexmlex (preferred)