SaveFocusedXYE mangles data columns

Bugs
1

Expected behavior

Text file written with XYE format…

  495.72436        0.00000000        0.00000000
  495.97222        0.00000000        0.00000000
  496.22020        0.00000000        0.00000000
  496.46831        0.00000000        0.00000000
  496.71655        0.00000000        0.00000000
  496.96491        0.00000000        0.00000000
  497.21339        0.00000000        0.00000000
  497.46200        0.00000000        0.00000000

Actual behavior

Oh No!

  519.83329        0.00000000        0.00000000
  520.09321        0.43168354        0.61049276
  520.35325        0.44957668        0.63579745
  520.61343        0.00000000        0.00000000
  520.87374        0.20619035        0.29159575
  521.13418      217.30827084 37635096.50433181
  521.39474        0.00000000        0.00000000
  521.6554471598028800.000000008845396037556458.00000000
  521.91627        0.00000000        0.00000000
  522.17723   462333.1250000080234663609.31842041
  522.43831        0.00000000        0.00000000

Steps to reproduce the behavior

Load EventWorkspaces…
Rebin to Workspace2Ds…

# smooth incoherent
if verbose: print('>>> smoothing incoherent data ...')
SmoothData(
    InputWorkspace='incoherent',
    OutputWorkspace='incoherent',
    NPoints=101
    )
ReplaceSpecialValues(
    InputWorkspace='incoherent',
    OutputWorkspace='incoherent',
    NaNValue=0,
    NaNError=0,
    InfinityValue=0,
    InfinityError=0,
    SmallNumberValue=1e-03, # <-- I appear to have misread the docs here
    SmallNumberError=1e-03   # <-- I should have set 'SmallNumberThreshold' 
    )

# divide
if verbose: print('>>> normalizing data ...')
Divide(
    LHSWorkspace='coherent',
    RHSWorkspace='incoherent',
    OutputWorkspace='norm'
    )
ResetNegatives(
    InputWorkspace='norm',
    OutputWorkspace='norm',
    AddMinimum=False,
    ResetValue=0
    )
ReplaceSpecialValues(
    InputWorkspace='norm',
    OutputWorkspace='norm',
    NaNValue=0,
    NaNError=0,
    InfinityValue=0,
    InfinityError=0
    )

Platforms affected

Windows 10 / Python 3.8 / Mantid 5.0.0
Windows 10 / Python 3.8 / Mantid 5.1.1

Summary

I can work around this bug by masking values using ReplaceSpecialValues, but I still consider this a bug. It forces me to mask values larger than 9E+08 because of the fixed column width, and thus a user with this workaround might silently pass over data anomalies or quash other data problems and hide them as zeros (as I’ve written it here).

A simple solution would be to use scientific notation generally or for large numbers.

Happy scattering!

2

I’m always amazed how many different Save algorithms there are in Mantid. While I can reproduce the problem you are having with SaveFocusedXYE (where there aren’t any spaces b/w large data columns)…

using the algorithm SaveAscii seems to work! I used the separator: “Space” and got data like:

# X   Y   E
1
1 1.23457e+13 0
2 1.23457e+13 0
3 1.23457e+13 0

SaveAscii data like this can definitely be reloaded back into Mantid, but does this algorithm work for your further analysis outside of Mantid?

3

Thanks for the tip-- yes that should do it.

In my case “.xye” is the specific file extension for a three-column x-y-error ascii file recognized by the Rietveld software Topas.

On the topic of the milieu of Save functions, it would be convenient if they all supported user input for comment character. # is a classic but Topas uses the single quote ' to denote comment lines, and I’m currently making a second pass on SaveFocusedXYE written files to clean them up for my purposes. I see SaveAscii ticks this box as well.

Thanks!