DBASE IV - Concurrent APPEND FROM ARRAY = broken index

[scotx]

I'm currently experiencing a problem of broken index where a dbase IV application is being used by two users in a network.

Setup

There are scenarios where the 2 users each call up one of two different functions that both write to the same table. The one functionality, for example, writes 100 single records to the table in quick fire using the APPEND FROM ARRAY. The other functionality writes single records to the same table each triggered by the user and the record is also added using the APPEND FROM ARRAY.
No explizit record locking with APPEND BLANK and RLOCK

Result

Not all 100 records from the one user instance are being written successfully to the table
The referencing to this table gives wrong results that are then only fixed when the index is recreated.

When the two functions run for themselves with no concurrent access then the written records are always correct.

Note

I'm currently trying to build a test tool to reproduce this effect because I can't send the originally code. That way I'll be able to produce a focused vdos log file.

[Jos]

Interesting, looks like a caching problem. Index pages being updated by the second user, based on a local outdated copy (not matching the actual pages on disk).

vDos doesn’t cache disk operations, so dBase would be to blame?

Opportunistic locking by the server seems non-relevant since that proved only problematic with mixing XP machines in specific situations.

I don’t think a log file will be informative, it doesn’t show the logic behind reading from and writing to disk.
A dedicated test tool to reproduce this anomaly would certainly be helpful.

Jos

[scotx]

Thanks Jos, for the first assessment.

The previous logic of performing an APPEND BLANK to the table and then trying to rlock() the record was not 100% successful either. Looking to test out dbase V, once I have it installed but I'm not very hopeful at the moment.
Will post again when I've got my test tool ready.

Scot-X

[Jos]

I’m no dBase programmer, but the sequence APPEND BLANK - rlock() - update the record seems even more prone to errors that a single APPEND FROM.

The data file - certainly with APPEND - is more or less a simple flat sequential file.

An index file on the other hand is organized in logical pages. dBase has to modify one or more (up-to-date!) pages in one transaction.
First lock and read each page leading to the final page containing the actual index reference.
Second in reverse, modify and rewrite the pages that need to be updated, before unlocking one.

If there’s any caching and concurrent updating, the data file is relatively safe. A record would eventually be overwritten.
But the B-Tree structure defined in the index file will be corrupted quickly.

Jos

[scotx]

The dbase V test has shown that the problem still exists and therefore it does not seem to be a dbase IV specific problem.
The table that is concurrently being updated has 3 indexes. What seems to have shown to be a possible solution, is the artificially slowing down of the rapid fire APPENDS by introducing a 1 sec pause after each update.

The tests involved two clients that are each appending 100 records to the same table. Some tests in the past have thrown the error code 108 'File locked' and the APPEND is wrapped in an ON ERROR and always attempts a RETRY.

More tests lined up for next week.

Scot-X

[Jos]

I’m intrigued; dBase IV/V is still used by many. Among those will be heavy duty multi-user settings.

If the rather simple task of concurrently adding 100 records to a table by two instances is destined to fail, dBase would be useless for multi-user operation.

Error code 108 'File locked' will not be related; the database will be opened exclusively by the first dBase instance.

Jos

[scotx]

The plan is to run the dbase application on a virtualised (Proxmox) windows 2022 server, with the help of vDos. The 10 or so users will have their RDP sessions to the server, where they will run their copies of the dBase application. They're using the same dbase installation and the same copy of the dbase application.

We have not looked into the filesystem settings on the windows side, that might have influence on the dbase behaviour under vDos.

I've been surprised and irritated by the behaviour but have yet to find a plausible reason.

We've even been wondering how, on a lower level, the rlock() works.

What I can also say is that the table has over 300,000 entries, so that any index updating will have something to do. Though we have no feeling for the relation between number of records, number of indexes and concurrent access when APPENDING FROM ARRAY.

Scot-X

[Jos]

The Windows filesystem settings should be of no influence to dBase operations, it’s all local.

When you (a program) are going to modify a record, first a lock is requested for the part of the file containing the record. So others (temporary) cannot modify it concurrently.

The OS managing the drive maintains a list of granted record (file region) locks. Note, this doesn’t involve any disk operation, that file-offset-length list is in memory.
Once the lock request is granted, all file (even read) operations that include (part of) that region will be refused by the OS until a unlock of that region is requested.

That blocking also includes the one who requested the lock. So at first that locking mechanism seems rather useless; you can lock a record but then not access it anymore.
Furthermore, others also can’t. If one has locked a record, everyone in need of that has to wait until that lock is revoked.
Suppose it’s a stock item, you would be unable to retrieve its details like the description, prices, number in stock…

Therefor a database program doesn’t lock the record itself. Instead it adds a systemwide consequent large number to the offset, typically 2GB.
That will normally be greater than the actual file size, but the OS doesn’t care; it’s just a list of file-offset-length.

Suppose a record is located at offset 123,456 of the file, length 100 bytes. A lock would be requested for 100 bytes at offset 123,456+2,147,483,648.
The record itself remains accessible, though basically read-only since a further lock will of course fail.

A test program should reveal what’s going wrong with appending from array. dBase caching write operations with this specific command?

Jos

[Jos]

One explanation for dBase (incorrectly) caching file operations came to mind:

• You run vDos/dBase local at the server.
• dBase queries whether the drive or file is local or remote.
• vDos returns it’s local; dBase decides it’s safe to speed things up by caching.
• Updating/flushing the buffer(s) only when needed at reading/writing beyond the buffered file region.
• Eventually also forced every ? seconds.

Jos

[scotx]

Jan 28, 2024 13:56:14 GMT 1 Jos said:

One explanation for dBase (incorrectly) caching file operations came to mind:

• You run vDos/dBase local at the server. 
  Scot-X: Yes, data and application are on the same drive
• dBase queries whether the drive or file is local or remote. 
  Scot-X: Assuming I understand your question correctly, we have already checked what the dbase "network()" function returns, .T. (True)
        Or is there another way of checking what dbase has determined in terms of local or remote?
• vDos returns it’s local; dBase decides it’s safe to speed things up by caching.
  Scot-X Would vDOS log this kind of information?
• Updating/flushing the buffer(s) only when needed at reading/writing beyond the buffered file region.
• Eventually also forced every ? seconds.
  Scot-X: Where or who would define these seconds?
  
  Scot-X: The old application still runs on a Windows 2003 server and there, the application and data are on the same drive. No problems in the past. Whereby I have to be careful because the application version that we are testing has been developed further and therefore not 100% the same. We have not yet taken the new application and tried to run it on the old server because we want it run only on the new Windows 2022 server.

Jos

[Jos]

• I don’t know what network() exactly returns. Not a drive being remote (drive is no parameter). Instead probably the DOS Network Redirector component being installed; programs are able to access remote drives.
• The log file reports mostly errors and some extra information. vDos doesn't know if a program would be caching.
• If buffers are also periodically updated, dBase will determine at what interval. Makes sense because the data on disk is not up-to-date while it’s modified only in the buffer(s). But also risky, if the program or PC would crash. Furthermore it would explain adding a pause resulting in less errors, if that can’t be done by caching on its own.
• I don’t know what (extended) logic dBase would use to determine if caching is save. First of course the drive or file being local or remote. If local, could be a further check is made if the PC is acting as a server, and/or the DOS Network Receiver component is installed. Although not completely correct, I’ll let vDos return both conditions as true. Will send you that to test. It’s for now the most simple test, and shouldn’t hurt any program.

Jos

[scotx]

Thanks Jos,

received your file. I don't think I'll be able to test it today because I'm on the road this afternoon. Will definitely have the time tomorrow.
So if I understand you rightly, this test version of vdos now has predetermined return values with remote = True and ServerMode = True.

The network() function in dbase IV, according to the Borland book says, that a True or False is returned stating whether dbase is running in a network or not. Otherwise, very little information.


Scot-X

[scotx]

OK, couldn't wait after all and did a quick start of the application with the testversion of vdos to see what appears.

The first 4 lines of the vdos.log file state the following:

   vDos 2024.xx.xx
0.11 CodePage: 850
   C: =>  (Local) T:\zMUKL\CONFIG\vDOS-DESKTOP\
   T: =>  (Local) T:\
Does that mean that vDOS is seeing the drives as local drives, which is essentially correct. The windows C: is where vdos is installed, the windows T: drive is where dbase and the application are installed. With DOS we are only working with a DOS T: drive that maps to the Windows T: drive. A bit confusing at times.

Scot-X

[Jos]

vDos indeed determines both drives being local.
Don’t know, but you could at least omit Windows T: by using UNC paths?
And if vDos.exe is located in T:\zMUKL\CONFIG\vDOS-DESKTOP\, omit USE C: altogether?

I verified dBase calls DOS API 2Fh-B800h: NETWORK - INSTALLATION CHECK.
Ralf Brown's Interrupt List (https://www.ctyme.com/intr/rb-5244.htm) is often an extensive reference guide, but for this one merely states:

Return:
AL = status
00h     not installed, nonzero installed
BX = installed component flags (test in this order!)
bit 6   server
bit 2   messenger
bit 7   receiver
bit 3   redirector
bit 1   LANPUP (LANtastic 4.0)
Notes: This function is supported by PC LAN Program, LAN Manager, LANtastic, NetWare Lite, SilverNET, 10NET, etc.. LANtastic and NetWare Lite use only BL for the return value, preserving BH; LAN Manager and DOS LAN Requester return BH=00h. This permits differentiation between those two groups by setting BH to a nonzero value before the call and checking its value on return.


vDos only returns bit 3 set, I added bit 6 and 7. As said network() would reflect bit 3.

I could surely debug what dBase is doing after calling INT 2Fh-B800h.
But afraid it will first only store the returned value as-is, modified, and/or split at some memory address(es), and use that information at several moments later on. Complicating the debug process…
So if this doesn’t bring a solution, a second quick test would instead be to just report all drives and files as remote.

Jos

[scotx]

Hi Jos, thanks for the more detailed information. The following preparation has been completed before the functional tests take place:

The updated autoexec.txt is now (first few lines):

rem This is essentially the DOS autoexec.bat.
rem =========================================

# Lines in this section will be run at startup.
# You can put your MOUNT lines here.
KEYB GR 850
REM Set the drive for the TEST
USE T: \\LENSRV04\TEST1

Produces the following vdos.log (first 4 lines):

    vDos 2024.xx.xx
0.08 CodePage: 850
    C: =>  (Local) T:\zMUKL\CONFIG\vDOS-DESKTOP\
0.09 T: => (Remote) \\LENSRV04\TEST1\
I'm not sure where the C: Drive is coming from because I'm not explizitlly creating one. The windows directory that is shown is where the vdos configuration is held (autoexec, config).

Another quick question, what happens to the vdos.log file when two windows users start the same application? Does the log file contain the information from both user sessions or does the second user win and overwrites the contents of the first user?

Will perform the functional tests tomorrow.

Scot-X