Adventures in File IO and Commander X16

[ZeroByte]

On 7/19/2022 at 6:43 PM, TomXP411 said:

The LFN and SA don't need to match. I tested this while helping ZeroByte troubleshoot his C code. Whatever's going on there is a bug in the C runtime, not CMDR-DOS. 



I actually tested explicitly using different Logical File Number and Secondary Address, and P works just fine. (ie: OPEN 3,8,5). 

I guess I should've been more clear that this was in C. Yes, as Tom mentions, he fired off a quick demo in BASIC that works using any LFN/SA.

Do you recall whether it worked for values not having bit 1 set?

[BruceMcF]

On 7/20/2022 at 9:43 AM, ZeroByte said:

I guess I should've been more clear that this was in C.

Ah, I don't program in C for the CX16 (C64, etc.) ... at one time, a quarter of a century ago, I programmed a brute force maximum entropy estimator in C, but that was on a two 3.5" disk drive MS-DOS machine (with hard drive via a parallel port adapter).

[ZeroByte]

So here's something that I'm not 100% on: what exactly IS the SA?

My current understanding is that SETNAM and SETLFS are basically glorified "accessor functions" - they literally do nothing but store the values in some ZP locations. So essentially, those two functions are "arguments" for whatever upcoming channel-related command is about to be issued (open, load, save, chkin, etc)

Thus - SA is essentially a general-purpose "argument slot" whose function/encoding depend upon whatever command is actually about to get called. (e.g. LOAD uses it to determine relocate (bit 0) and headerless (bit 1))

Somehow in practice, its functionality seems to transcend this. It's also interesting how the channel number has significance as well.

What a mess.

[TomXP411]

On 7/21/2022 at 7:49 AM, ZeroByte said:

So here's something that I'm not 100% on: what exactly IS the SA?

It's entirely implementation dependent: it will be different for a disk drive, a printer, the serial port, and whatever other devices you can imagine. 

Specifically, with the disk drive, there are 3 reserved values.

15 is the command channel. Sending data to this channel issues DOS commands, rather than writing to disk. 

0 is used for LOAD

1 is used for SAVE

I'm going to circle back around to my earlier statement, I think either your code or the C library you're using is either mixing up or swapping the channel number and secondary address. You should probably audit the code and confirm the correct usage of the carry flag, and the correct values in .A, .X, and .Y in all of the KERNAL calls. I don't see any of the problems you're describing when using BASIC, so your confusion is likely due to bad code, not the KERNAL API.

[ZeroByte]

okay - so having followed the code in the kernal source, it appears that there's a concept I was not yet aware of...

I think LOAD is an exception to the rule, but in general, channel-related commands such as open/close/chkin/etc.... those all consider SA to be broken into two subcomponents:

high nybble = command, low nybble = channel.

The commands like open seem to manipulate the command portion of this.... having to do with talk / untalk / listen / unlisten - commands for the serial bus (or the CMDR-DOS pretending to be a serial device in the case of X16 on device 8.)

Man, this onion keeps having more layers. That's why I wasn't understanding what you were saying, @TomXP411... I was thinking LFN was the channel, essentially. I know it means "logical file number" - but now I have a new question - can LFN be literally any byte the program wants to use? Could you use LFN = 72 if you want?

 

[BruceMcF]

On 7/21/2022 at 3:27 PM, ZeroByte said:

okay - so having followed the code in the kernal source, it appears that there's a concept I was not yet aware of...



I think LOAD is an exception to the rule, but in general, channel-related commands such as open/close/chkin/etc.... those all consider SA to be broken into two subcomponents:



high nybble = command, low nybble = channel.



The commands like open seem to manipulate the command portion of this.... having to do with talk / untalk / listen / unlisten - commands for the serial bus (or the CMDR-DOS pretending to be a serial device in the case of X16 on device 8.) ...

Yes. Remember that the LFN is information for the KERNAL, which maintains the list of open channels, and the secondary address that goes out on the IEC bus is for the device that you are talking to. For a 1541 (and similar) device, a secondary address of 0 and 1 are binary load and save, secondary address of 15 is the command channel, and secondary addresses between 2 and 14 are intended for sequential or relative files.

Now, IIRC, when doing a LOAD, the actual IEC bus SA (which is indeed in the lower nybble of a byte sent on the IEC, hence the limit to the range of 0-15) is implied by the KERNAL call, so the KERNAL call overloads that with information on how to do the load ... whether to load at a specified address or at the address that the binary file was originally SAVED from. AFAIR, the Commander X16 extends that to allow specifying the target starting bank.

The LFN can indeed be larger than 14, but AFAIR, most people did not see any need to have a LFN that was different from the SA when opening sequential or relative files.

[TomXP411]

On 7/21/2022 at 12:27 PM, ZeroByte said:

okay - so having followed the code in the kernal source, it appears that there's a concept I was not yet aware of...



I think LOAD is an exception to the rule, but in general, channel-related commands such as open/close/chkin/etc.... those all consider SA to be broken into two subcomponents:



high nybble = command, low nybble = channel.



The commands like open seem to manipulate the command portion of this.... having to do with talk / untalk / listen / unlisten - commands for the serial bus (or the CMDR-DOS pretending to be a serial device in the case of X16 on device 8.)



Man, this onion keeps having more layers. That's why I wasn't understanding what you were saying, @TomXP411... I was thinking LFN was the channel, essentially. I know it means "logical file number" - but now I have a new question - can LFN be literally any byte the program wants to use? Could you use LFN = 72 if you want?

I'm also a little surprised, looking at this. It turns out that Position uses the Secondary Address, not the file number. 

So this should work:

OPEN 1,8,2,"TEST FILE,S,R"

OPEN 15,8,15,"P"+CHR$(2)+CHR$(64)+CHR$(0)+CHR$(0)+CHR$(0)

This will not work (using file number in the P command):

OPEN 1,8,2,"TEST FILE,S,R"

OPEN 15,8,15,"P"+CHR$(1)+CHR$(64)+CHR$(0)+CHR$(0)+CHR$(0)

and this will not work (using device number in P command):

OPEN 1,8,2,"TEST FILE,S,R"

OPEN 15,8,15,"P"+CHR$(8)+CHR$(64)+CHR$(0)+CHR$(0)+CHR$(0)

So where we all got it wrong was assuming "channel" meant the LFN, or Logical FIle Number, aka the lfn in OPEN lfn, device, sa

That is why you thought that the LFN and the SA needed to match: they did not need to match, but if you were using the LFN with the Position command, it would obviously not work. 

Here is the test program I used to figure this out. It tests all permutations of LFN (up to 15) and SA. You can change the FOR loops to test other values and change the SA in 170 to LFN to see what happens when you try to use LFN in the P command. 

 

10 OPEN 8,8,8,"TEST DATA,S,W"

20 FOR I=1 TO 255

30 PRINT#8,CHR$(I);

40 NEXT

50 CLOSE 8

100 PRINT "LFN","SA","TEST","GOT"

110 FOR SA = 2 TO 14

120 FOR LN = 1 TO 15 : REM CAN BE 1 TO 255, CAPPING AT 15 FOR BREVITY

140 PRINT "    ","    \X91"

150 PRINT LN,SA,

160 OPEN LN,8,SA,"TEST DATA,S,R"

170 OPEN LN+1,8,15,"P"+CHR$(SA)+CHR$(64)+CHR$(0)+CHR$(0)+CHR$(0)

180 GET#LN,A$

190 IF A$=CHR$(65) THEN PRINT "PASS\X91" : GOTO 210

200 PRINT "FAIL ", ASC(A$), : DOS

210 CLOSE LN

220 CLOSE LN+1

230 NEXT

240 NEXT

 

 

[BruceMcF]

On 7/22/2022 at 12:34 AM, TomXP411 said:

I'm also a little surprised, looking at this. It turns out that Position uses the Secondary Address, not the file number.

Yes, that much is the same as the 1541 Relative Files "P" command. From "Relative files for the VIC-20 and Commodore 64" by the estimable Jim Butterfield,

Quote

Now, we send our "position" command down the command channel. To identify to the disk which file we want to position, we use the secondary address. For our relative file in progress, that would be 2. That's important: secondary address, not logical file number. Now, another thing about the disk: it likes to see you add 96 to the secondary address, so we should send 98.

After all, the relative file positioning is being done by the 1541 (or 1571 or 1581) ... the computer is simply sending a text string on the command channel. And the IEC storage device only knows about the SA ... it has never been told about the LFN, which is a record keeping ID on the computer it is talking to. It is receiving information on it's "15" channel about a file it is storing that has been opened on one of its channels from 2-14.

This is why the fig-Forth I was using back in the day most of the time felt so much more responsive than using disk file I/O in other programs ... most of the file I/O was in 1K BLOCKS, in a relative file, so, suppose that a BLOCK buffer was full and needed to be saved and replaced with another block ... it would seek to the block presently in the buffer, write the block, seek to the desired block, read the block, and then return ... which is only 2KB going over the slow IEC interface. In many cases, when interpreting source from block, the source was already in a block buffer, so it wouldn't even have to go to disk at all.

 

[ZeroByte]

Having gone a few layers down the onion in Kernal, I can tell you that what LOAD does is skip calls to OPEN and CHKIN in favor of doing the low-level task itself - it sends a temporary SA of $60 to the disk drive (or in the case of X16, it talks to itself like Gollum in LotR with one half of the convo being "kernal" and the other half being "dos") and sends the TALK / UNTALK type commands itself. (CHKIN usually does this)

I think the use of the term "channel" is what I got tripped up over. Keeping in mind the system that X16 is emulating, it makes sense that it would be the SA and not the LFN - having learned a thing or two about how channels work.

I'm still not all there yet, but I've made a good bit of progress towards full understanding. (hence the name of this thread)

My next issue arose yesterday while attempting to make an assembly program to do essentially the BASIC test tom posted, I ran into the issue that I'm used to using CHRIN / CHROUT where these use the default device (i.e. the keyboard/screen) and am not entirely certain how to revert to these once I've used non-default IO devices. (hopefully without having to call CLRCHN which closes all open files)

 

[BruceMcF]

On 7/22/2022 at 12:43 PM, ZeroByte said:

Having gone a few layers down the onion in Kernal, I can tell you that what LOAD does is skip calls to OPEN and CHKIN in favor of doing the low-level task itself - it sends a temporary SA of $60 to the disk drive (or in the case of X16, it talks to itself like Gollum in LotR with one half of the convo being "kernal" and the other half being "dos") and sends the TALK / UNTALK type commands itself. (CHKIN usually does this)

This makes sense. As explained in Micheal Steil's Commodore KERNAL History, the earliest version of the KERNAL was to run BASIC on a PET 2001 system, with six "KERNAL" calls -- OPEN, CLOSE, LOAD, SAVE, VERIFY and SYS -- that are not really assembly language subroutines, but implement the actual BASIC keyword, with that version of Basic for PET assembled to just use the correct jump in the jumptable. So when the KERNAL code for the VIC-20 was revamped to make it more useful for assembly language programming, it makes sense that they probably just took the appropriate parts of the earlier CBM written LOAD and SAVE keywords and used for the more general purpose KERNAL LOAD and SAVE routines.

Quote

I think the use of the term "channel" is what I got tripped up over. Keeping in mind the system that X16 is emulating, it makes sense that it would be the SA and not the LFN - having learned a thing or two about how channels work.

Yes ... starting from PC-DOS, the assumption has been that the system is running the device, and so we now are used to an external device getting an interface driver that runs to make it "look like" the system itself is running the device. The PET / VIC-20 / C64 / C128 family was the other way around, with the assumption that the device you are talking to is an external device, and when its an internal device, either the keyboard, screen, Datasette or User Port serial port, it's set up "as if" the system CPU is running both sides of the process.

Quote

I think the use of the term "channel" is what I got tripped up over. Keeping in mind the system that X16 is emulating, it makes sense that it would be the SA and not the LFN - having learned a thing or two about how channels work.

Yes ... AFAIU, the "LFN" was originally just the number you put after the "#" in Basic Keywords that could take input from somewhere other than the keyboard and put output somewhere other than the screen, and then made a little more general to better support assembly language programs. It was at that point that the "low level" serial IEC calls were added -- TALK, LSN, UNLSN, UNTLK, IECOUT, IECIN, SETTMO, TKSA, SECOND.

Quote

I'm still not all there yet, but I've made a good bit of progress towards full understanding. (hence the name of this thread)



My next issue arose yesterday while attempting to make an assembly program to do essentially the BASIC test tom posted, I ran into the issue that I'm used to using CHRIN / CHROUT where these use the default device (i.e. the keyboard/screen) and am not entirely certain how to revert to these once I've used non-default IO devices. (hopefully without having to call CLRCHN which closes all open files)

If I am reading the KERNAL API correctly, I think it's CLALL that closes all open files, and CLRCHN only "clears" the input and output channels to their defaults. You wouldn't have to use CLRCHN itself if you WANT to close all of the files, because CLALL calls CLRCHN as part of the process.

IOW, IIRC, you can CLRCHN without closing the open files, use the default CHRIN and CHROUT, and then use CHKOUT and/or CHKIN to switch back to other, still open, input or output channels. It's been a while since I've used the KERNAL at that level, but I don't see any other way to get input and output back to the defaults, since there isn't any LFN to use.

As far as how the C language library IMPLEMENTS that ... ¯\_(ツ)_/¯ I don't know 6502 C. I know 6502 Forth. 6502 Forth been berry, berry good to me.