LoadAscii fails to recognize "new" IDFs

General
1

The algorithm in LoadAscii2 seems to have some problems (or seems to have at least uncovered some problems) related to reading instrument definition files. In an attempt to not-reinvent-the-wheel, I have created a local directory in the Mantid installation directory, and then I have copied over various existing instrument definition files (and the Schema directory, as well).

Symptom:
(1) If I merely copy the IDF from the ‘instrument’ folder to my local folder, when I use Workspace → Load → File, the IDF loads correctly.

(2) If I make changes to the IDF, as long as I maintain the correct format as defined under the Instrument Definition File wiki, I am able to create any geometry I like, save the file (with the original filename) and the IDF loads correctly.

(3) If I attempt to copy my newly created IDF (the one from (2) above) into a new blank text file, save it as xml, and attempt to load it, I receive the error code below.

(4) If I attempt to rename the original IDF (from (1) above) to anything else, I receive the error code below.

Error Code:
Failed to read as ASCII this file: 'D:\MantidInstall\local_inst\something.xml, error description: No valid data in file, check separator settings or number of columns per bin.
Error in execution of algorithm LoadAscii:
Failed to recognize this file as an ASCII file, cannot continue.
Error in execution of algorithm Load:
Failed to recognize this file as an ASCII file, cannot continue.

What I have Done:

  • I converted the Windows CR/LF to LF. No differences.

  • I converted \tabs to spaces. No difference.

  • I used the Windows command line utility [fc original.xml renamed.xml] and [fc /b original.xml renamed.xml]. Both indicated the files are identical (/b == binary file compare).

I am using Windows 10 (but also on my Mac running Yosemite) and have received the errors on both. These errors occurred on both 3.10 and 3.8 versions of Mantid (Windows) and version 3.8 (OSX).

Looking at the source, it seems that header parsing throws its exception in the vicinity of line 400 of LoadAscii2.cpp; ‘numCols’ isn’t getting the proper value. I cannot understand why renaming a file locally would cause the problems I’m encountering, however…

Is there some sort of registry where IDF’s names and versions get stored? Any advice?

Peter

2

Solution:
Either

  • Make sure the file name follows the format “[instrument_name] Definition [anything else you want].xml”
  • Use the LoadEmptyInstrument algorithm directly

Explanation:

The Load algorithm (which is what clicking on Load->File uses) tries to automatically decide on the specific file loader to use based on the contents of he file it is trying to Load.

Here the problem is that in one case the Load is picking LoadEmptyIntrument correctly to do the job, and in the other it is picking LoadAscii which expects columns of ascii data.

The code that selects which file to load can be based on that filename and the start of the file contents, but in this case it is just using the filename. Specifically it is checking for a text file, with an .xml extension, and the word “Definition” in the filename.

  if (descriptor.isAscii()) // Only consider an Ascii file
  {
    // Filename must contain "Definition"
    std::string::size_type stripPath = filePath.find_last_of("\\/");
    if (stripPath == std::string::npos)
      stripPath = 0;
    if (filePath.find("Definition", stripPath) != std::string::npos) {
      // We have some confidence and it depends on the filetype.
      if (descriptor.extension() == "xml") {
        confidence = 80;
      } else {
        confidence = 20;
      }
    } // Has "Definition"
  }   // Ascii file

The need for Definition in the filename is a holdover from when we first decided that instrument definitions should follow a standard format of instrument_name Definition.xml to allow us to find the correct file quickly from the filename alone. This expanded later to allow for multiple files per instrument, but the files still need to include the instrument name at the start and the word “Definition” as well as the xml extension.

The other option is to bypass the selection of which loader to use by using the algorithm LoadEmptyInstrument directly.

3

In answer to your other question of Where does Mantid load instrument definition files from when loading data, well it can be a bit complicated, because Mantid has a list of location to use, partially to handle multi user installations, but also to handle downloaded instrument files that have updated since your installation of Mantid.

So let’s start by saying this only applies when you are loading a data file and Mantid it trying to decide which instrument file it should load to match the data.

For any loaded data file you can always see where it loaded the instrument from by expanding the Workspace in the workspaces list and looking at the instrument.

Instrument Location.png705×286 9.8 KB

In selecting which file Mantid follows this in order:

  1. If the file is a Nexus file and has an embedded Mantid instrument definition it will use that
  2. Mantid will search all of it’s instrument definition directories for files that match the name of the instrument (and the word Definitionn and “.xml”)
  3. Mantid will then pick the file that has the Valid_From date that best matches the start date in the data file.
  4. You can use the python command below to see which directories Mantid is searching on your system.
print ConfigService.getInstrumentDirectories()
4

Thanks Nick,
I think my problem was that my understanding of the GUI was not complete. The second option was the golden ticket. As you pointed out, there is an option to run algorithms directly (under, oddly enough, Algorithms), and after I located LoadEmptyInstrument I was able to load the IDF directly and with no trouble whatsoever.

For the edification of anyone reading this in frustration, the LoadAscii algorithm executed by running Load->File on a home-built IDF did not work until LoadEmptyInstrument was called. Once the IDF was loaded using the empty instrument algorithm, it then became possible to simply run the Load->File->MyIDF.xml (replace MyIDF with whatever).

It seems there is more than one road to Rome…

Peter