Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DbfDataReader use old DBF specification #35

Open
s-s-a opened this issue Feb 24, 2019 · 5 comments
Open

DbfDataReader use old DBF specification #35

s-s-a opened this issue Feb 24, 2019 · 5 comments

Comments

@s-s-a
Copy link

s-s-a commented Feb 24, 2019

Below some quotes from Microsoft Visual Foxpro documentation.
Very impotant byte for correct table encoding - 29 | Code page mark

Table Header Record Structure
Byte offset | Description

  • 0 | File type:
    0x02 FoxBASE / dBase
    0x03 FoxBASE+ / FoxPro /dBase III PLUS / dBase IV, no memo
    0x30 Visual FoxPro
    0x31 Visual FoxPro, autoincrement enabled
    0x32 Visual FoxPro, Varchar, Varbinary, or Blob-enabled
    0x43 dBASE IV SQL table files, no memo
    0x63 dBASE IV SQL system files, no memo
    0x83 FoxBASE+/dBASE III PLUS, with memo 0x8B dBASE IV with memo
    0xCB dBASE IV SQL table files, with memo
    0xF5 FoxPro 2.x (or earlier) with memo
    0xFB FoxBASE (?)
  • 1 - 3 | Last update (YYMMDD)
  • 4 – 7 | Number of records in file
  • 8 – 9 | Position of first data record
  • 10 – 11 | Length of one data record, including delete flag
    **- 12 – 27 | Reserved
  • 28 | Table flags:
    0x01   file has a structural .cdx
    0x02   file has a Memo field
    0x04   file is a database (.dbc) This byte can contain the sum of any of the above values. For example, the value 0x03 indicates the table has a structural .cdx and a Memo field.
  • 29 | Code page mark
  • 30 – 31 | Reserved, contains 0x00
  • 32 – n | Field subrecords The number of fields determines the number of field subrecords. One field subrecord exists for each field in the table.
  • n+1 | Header record terminator (0x0D)
  • n+2 to n+264 | A 263-byte range that contains the backlink, which is the relative path of an associated database (.dbc) file, information. If the first byte is 0x00, the file is not associated with a database. Therefore, database files (.DBC) always contain 0x00.**

Field Subrecords Structure

Byte offset Description
0 – 10 Field name with a maximum of 10 characters. If less than 10, it is padded with null characters (0x00).
11 Field type:
W   -   Blob
C   –   Character
C   –   Character (binary)
Y   –   Currency
B   –   Double
D   –   Date
T   –   DateTime
F   –   Float
G   –   General
I   –   Integer
L   –   Logical
M   –   Memo
M   –   Memo (binary)
N   –   Numeric
P   –   Picture
Q   -   Varbinary
V   -   Varchar
V   -   Varchar (binary)
Note For each Varchar and Varbinary field, one bit, or "varlength" bit, is allocated in the last system field, which is a hidden field and stores the null status for all fields that can be null. If the Varchar or Varbinary field can be null, the null bit follows the "varlength" bit. If the "varlength" bit is set to 1, the length of the actual field value length is stored in the last byte of the field. Otherwise, if the bit is set to 0, length of the value is equal to the field size.
12 – 15 Displacement of field in record
16 Length of field (in bytes)
17 Number of decimal places
18 Field flags:
0x01   System Column (not visible to user)
0x02   Column can store null values
0x04   Binary column (for CHAR and MEMO only) 0x06   (0x02+0x04) When a field is NULL and binary (Integer, Currency, and Character/Memo fields)
0x0C   Column is autoincrementing
19 - 22 Value of autoincrement Next value
23 Value of autoincrement Step value
24 – 31 Reserved
@s-s-a
Copy link
Author

s-s-a commented Feb 24, 2019

Tip

You can use the following formula to return the number of fields in a table file: (x – 296/32). In the formula, x is the position of the first record (bytes 8 to 9 in the table header record), 296 is 263 (backlink info) + 1 (header record terminator) + 32 (first field subrecord), and 32 is the length of a field subrecord.

@s-s-a
Copy link
Author

s-s-a commented Feb 24, 2019

Code Pages Supported by Visual FoxPro

Code page Platform Code page identifier
437 U.S. MS-DOS x01
620 * Mazovia (Polish) MS-DOS x69
737 * Greek MS-DOS (437G) x6A
850 International MS-DOS x02
852 Eastern European MS-DOS x64
857 Turkish MS-DOS x6B
861 Icelandic MS-DOS x67
865 Nordic MS-DOS x66
866 Russian MS-DOS x65
874 Thai Windows x7C
895 * Kamenicky (Czech) MS-DOS x68
932 Japanese Windows x7B
936 Chinese Simplified (PRC, Singapore) Windows x7A
949 Korean Windows x79
950 Traditional Chinese (Hong Kong SAR, Taiwan) Windows x78
1250 Eastern European Windows xC8
1251 Russian Windows xC9
1252 Windows ANSI x03
1253 Greek Windows xCB
1254 Turkish Windows xCA
1255 Hebrew Windows x7D
1256 Arabic Windows x7E
10000 Standard Macintosh x04
10006 Greek Macintosh x98
10007 * Russian Macintosh x96
10029 Macintosh EE x97

@chrisrichards
Copy link
Member

Are you able to supply one or more sample files for the Visual FoxPro (0x32) file type with the Varchar, Varbinary, or Blob-enabled columns?

@pyt0xic
Copy link

pyt0xic commented May 28, 2022

Are you able to supply one or more sample files for the Visual FoxPro (0x32) file type with the Varchar, Varbinary, or Blob-enabled columns?

I would be able to supply some if you would like @chrisrichards? I work with them on a daily basis so I am currently sitting with hundreds of gigs of VFP tables.

@chrisrichards
Copy link
Member

@pyt0xic yes please, if you could send some files that would be great, thanks! Would you be happy for the files to be added to the repo?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants