All Text File Schema Definition files will adhere to some basic rules about their structure, character encoding and so forth. Not following any of these will cause the Text File Schema Definition file to be considered invalid by all validity checking and parsing tools.
1. Schema File
The computer file that contains the Text File Schema data will be referred to as the "Schema File".
2. Text File
The computer file that contains the textual data that is being decribed by the Text File Schema data will be referred to as the "Text File".
3. Character Encoding
4. Media Types
The Media Type for a Schema File when it is transmitted is currently "application/x-tfsd".
5. Schema File Filename Extension
The Schema File will always be saved to disk with the filename extension of ".tfsd".
6. Schema File Type
The Schema File type will always be a delimited text file, with a Newline terminating each line.
7. Records and Fields
Each line of the Schema File is referred to as a "Record". Each delimited section of a line is referred to as a "Field".
8. First Record
The first record of the Schema File will always be the Control record.
9. Last Record
The last record of the Schema File will always be the Summary record.
10. Record Order
11. Parent and Child Relationships
All records in the Schema File will have a Parent Record. The Control record is it's own parent, because it is the primary record of the schema. The Meta record has an implicit parent, the Control record. For the List, Condition, Rule and Summary records, their implicit parent is the Meta record. For the Record, Field and Value records, the parent is explicitly given in their ParentID fields.
12. Schema Field Delimiter
The schema field delimiter will always be the thirteenth character in the Schema File. It must be a valid printable character whose Unicode code point is one of these: 21hex to 2Chex, 2Fhex, 3Ahex to 40hex, 5Bhex to 60hex, 7Bhex to 7Ehex.
13. Fields as a Key-Value Pair
Each field is a key-value pair. The Schema File uses a single character to separate the key and the value.
14. Key Case Sensitivity
The key can be in any case. It is NOT restricted to what is shown in these definitions.
15. Schema Key-Value Separator
The schema key-value separator will always be the fifth character in the Schema File. It must be a valid printable character whose Unicode code point is one of these: 21hex to 2Chex, 2Fhex, 3Ahex to 40hex, 5Bhex to 60hex, 7Bhex to 7Ehex.
16. Field Order
With the exceptions of the Type and ID fields, any other Schema File field can be in any order in the record. The Type field must always be the first field in the record and the ID field must always be the second field in the record.
17. Field "Has Data" Property
A Schema File field's "Has Data" property can be either "Always", "Optional" or "Conditional".
This indicates that the field must always have a non-empty value.
This indicates that the field is allowed to have an empty value.
This indicates that the field must follow the rules given in the record's Notes section.
18. Field Presence
If a Schema File field's "Has Data" property is "Optional" or is "Conditional" and the fields condition allows it to not have data, then the field does not need to be present in the record. When the Schema File is processed, any missing fields will be assumed to have an empty string as their value.
19. Trailing Delimiters
It is permissible to have multiple trailing delimiters in order to make all records appear to have the same number of fields. This is not a requirement, however.
20. Field Enclosing
The entire field must be enclosed by the Control.SchemaEnclosureCharacter if it contains the schema field delimiter or the Control.SchemaEnclosureCharacter. Under no circumstances can the Control.Type be enclosed.
21. Double Escaping
The Control.SchemaEnclosureCharacter will be double escaped if it is in a value.
22. Character Exclusivity
The schema key-value separator, schema field delimiter and Control.SchemaEnclosureCharacter must all be different from the others.
23. NULL Values
There is no representation of a NULL value, only empty strings.
24. Data Types
These data types are only for a Schema File. They are not to be confused with what is allowed in a Data File as defined by the DataType field.
Any alpha-numeric character or set of characters. The maximum length of the String shall not exceed 255 characters.
24.B. String List
Any String from the provided list of Strings.
Any numeric value, written in the Hindu–Arabic Numeral System using Hindu–Arabic numerals. The decimal separator will be the character whose Unicode code point is 2Ehex "Full Stop".
24.D. Number List
Any Number from the provided list of Numbers.
Any Number, without any decimal separator or fractions.
24.F. Integer List
Any Integer from the provided list of Integers.
Any positive Integer, including zero.
24.H. Unique Count
Any Count that is not repeated for any instance of the field defined with this data type.
Any Count, written in hexadecimal form.
24.J. Hexadecimal List
Any Hexadecimal from the provided list of Hexadecimals.
25. TFSD Field
In order to allow a data validation tool to be able to review a Text File against it's Schema File automatically, a Text File is allowed to have a special field called the TFSD Identification for this purpose. This consists of a single key-value pair, it's key is "TFSD", it's key-value separator is "=" and it's value is a publicly accessible URL that resolves to a valid Schema File. The URL must end with the characters ".tfsd". This is placed in any field in the Data. Any data validation tool must inspect the Text File looking for this field and use that Schema File to validate the Text File. The actual code may differ between validation tools to determine if this record exists, but a suggested methodology is given here.
25.A. Suggested Methodology
- Search for the string "TFSD=".
- Search for the first instance of the string ".tfsd" after the prior search.
- For every instance, see if the value can be resolved to a valid URL
- Attempt to load the URL as a Schema File.
- If valid schema data is found, use it as the Schema File.
- If valid schema data is not found, continue hunting in the Text File for another TFSD field.
- If no valid Schema Files are found in the Text File, request a Schema File from the user.
25.B. Sample Text File With A TFSD Field
HDR,SO12345,Billy Jones,20160605,UPSGRND,Happy Birthday!!,TFSD=http://www.acme.com/schemas/sample.tfsd DTL,DDF34,12 LOT,16136ABF,6 LOT,16137PDG,6 DTL,EPS12,12 LOT,16124RFG,4 LOT,16128JJH,6 LOT,16128LVW,2 NTE,,PACK WITH GREEN PEANUTS SUM,24