19. January 2009 20:01
I'm working on a program that must read and interpret text files produced by a specialized instrument. I have a document that describes the fields contained in the file system (or rather a set of files produced by the system.) The main file is a sort of dictionary that explains the collection of data files, linking data to master information. The file documentation is quite good, and one of our engineers has produced a wonderful spreadsheet of the file format that must be read and processed.
The documentation ties all information together quite nicely, and I was able to write special reader components that implement the IDataReader interface. Our sample files processed beautifully, giving us excellent data. Then I received more data from the field:
- Real world files, with hand entered data.
- Variations occurred between different environments.
- All of a sudden, files were missing from the collection.
- Should I throw an exception, or should I ignore the problem?
- Relationships would break if I ignored the problem.
- Fields in the documentation were not used in quite the same way between different measument environments.
- Out of the blue, required fields appeared to be optional, and data files changed format, depending on how the collection was performed.
My project was supposed to process two data file types. I now have about a dozen different file types. Engineers discovered that files that were considered the same type were actually quite different in the details. Different enough that I had to perform some major rewriting deep within the project schedule.
So what questions should have been asked?
- We asked how many file types needed to be parsed.
- We asked for sample date.
- We asked for internal file documentation.
Yet, the helpful and excellent answers we received weren't enough to prevent some costly mistakes from being made. Unneccessary mistakes. I'm not sure what we should have done differently. Should the customer bear the full cost of the problem? Maybe. It's not my call. I just produce the deliverable.
Please let me know your thoughts. How do you specify such things?