Hardware Expansion and Driver Handling

[Lorin Millsap]

Ok, many points on this subject are spread across multiple other threads and I’d like to have a specific thread where these ideas can be hashed out and discussed and pros and cons of various approaches can be addressed without distracting other threads.  I suppose I should start by explaining in as clear of terms as possible what the design approach has been as it appears there is confusion on this topic.  How the IO logic is broken down in simple terms. There are a total of 8 active low select lines. Three of these lines are used by integrated hardware. The first handles both ViA chips, the second handles video, and the third handles audio. This leaves 5 lines free to support expansions.  Each IO line or range spans 32 bytes of addressable space. Most chips will use far less than this, but in the even a chip used more you could utilize more by occupying more than one select line.  This is large enough for IO devices but not really large enough for a ROM, at least not a directly mapped ROM.  Now on to how these IO select lines are routed. They are all routed to every expansion slot. In other words the system has no distinction for which slots perform which functions. As far as the X16 is concerned all expansion slots are essentially equal. The upside of this is it allows expansion cards to be configured in a flexible way, but the downside is it does leave room for configuration errors. A cards physical slot has no bearing on how it gets mapped to memory.  So how then are we proposing that cards get mapped to the IO lines. By physical jumpers or dip switches on the card. What happens if two cards have the same address? There will be bus contention and it won’t work. Real damage is unlikely, but that hardware simply won’t work until the conflicting lines are corrected. This is a problem PCs did have however we have way simpler select options, you only have 5 potentially valid ranges so even if you tried by trial and error it wouldn’t take long to get a valid configuration.  So the next major issues that can come up are IRQ handling and DMA.  On the X16 there is no IRQ prioritization in hardware. There are two types of IRQ events. The first and more common is a standard IRQ which is invoked in hardware by pulling the IRQ line low. This will cause the CPU to complete its current instruction, then push its registers to the stack and then jump to the IRQ vector. It will then determine what caused the IRQ in software. This is the way the commodore machines handled it and even though you spend CPU cycles polling a few registers to determine what caused the interrupt, it for the most part works and is still handled faster in most cases than other CPUs of the era. IRQ events occur on a first come first served basis. In other works if an IRQ is triggered while an IRQ event is being handled the second IRQ is not handled until the first one is completed. Also the CPU can be set to ignore this type of interrupt.  The second type of IRQ is a NMI or Non-Mashable Interrupt. This uses a separate NMI line and this type always gets priority. Even if the CPU is set to ignore interrupts, NMI will still interrupt normal operation. Even if the CPU is handling a standard interrupt, a NMI will take precedence.  I’ll skip DMA for now.  So now that I’ve explained the basic architecture that allows us to look at the issues you might run into and how you can best handle them.  1. Hardware can technically be at any one (or more) of 5 address ranges. 2. Hardware can potentially trigger IRQ or NMI events3. Standardized ways of utilizing certain types of hardware and making it accessible to software may be desired. 4. The actual code to handle the hardware needs to reside somewhere in memory and needs to do so in a way that doesn’t conflict with other drivers. This code probably needs to have some degree of relocatability and any vector tie ins need to be configured based on where the code is located.  So the approaches may vary based on what you are trying to do. So I encourage civil discussion. We don’t want any complex or severe changes. But that doesn’t mean we can’t discuss options and possible solutions.  My primary approach is that as things develop we will try to get an auto boot feature working. The details on that haven’t been sorted yet, but I’m of the opinion that a type of config file written in plain text similar to dos allows a good balance between ease of use and flexibility. I’m in the fan base too that these config files are essentially BASIC while functioning like a DOS batch file. So you can potentially do some pretty powerful things without having to know assembly. These files can be configured to load drivers and even pass arguments in some cases. So my thinking is that a driver can be loaded and told where it’s hardware is located. The driver may load a config file or something, there are several ways it could be handled, but the main things is where the hardware is located since that also affects where the IRQ hooks need to be if there are any, and can provide a clean way that hardware can be accessed by relevant programs without having to build drivers into the programs themselves.   Sent from my iPhone using Tapatalk

[Lorin Millsap]

So obviously for hardware you may need to have an installer of some sort. It might be possible in some cases to have the installer try to auto detect the hardware, but it’s probably fine to just have the installer ask how you’ve set your jumpers or dip switches.

Then it could say inject some lines into your config file and place the files where they need to be. I’m not opposed to this option as it is highly flexible while also being reasonably user friendly. It’s obviously not plug and play. But it’s not crazy difficult either.


Sent from my iPhone using Tapatalk

[Lorin Millsap]

On the subject of potentially having a ROM on the expansion cards, a potential solution could be bringing SPI or I2C into the expansion slot. Then you could place a ROM on that card but this alone does not fully address some of the issues so the code in this ROM would need to have enough flexibility to handle the issues.

The main issues
1. the code still needs to be able to account for the address the card has been assigned to. So you either need a way to assign what ID the card is or to auto detect the cards ID.
2. You cannot execute code directly from a serial ROM so it still needs to be copied to RAM in a way to doesn’t overwrite other important code.
3. The code has to set itself up or in order words it is self modifying code. It needs to basically load into a staging area, then execute and copy its routines to a safely assigned area and tie itself into the KERNAL vector table and tie in relevant IRQ.

This is all doable, but whether it’s located in a ROM on the card or gets placed on the SD card it still has to account for and dolce the same problems. Locating it on the SD card allows greater flexibility. Putting it on a built in ROM kind of black boxes the code to a degree. Yes it may have a degree of plug and play ease if set up correctly, but it brings new challenges too.


Sent from my iPhone using Tapatalk

[m00dawg]

I should say I'm not really qualified to comment about the hardware ? But I do think the conversation should keep going. We chatted about this at length yesterday and I thought there were some good ideas, but also not sure how they would be implemented. Like if there could be a hardware standard of sorts that might include cards responding the same way to a special string of data put in the I/O spaces, one could probe for cards.

But at the same time, I think a config file would avoid this as you say and I kinda find it interesting a user given the X16 is now part Commodore and part PC DOS in a way ? Bonus points if the file is called config.sys ?

This led us to wondering what the concept of a "driver" would be for various cards and what that would mean. Like say if someone builds a GUS style soundcard that has it's own memory. One could then load a buncha high quality PCM data for triggering for "hardware accelerated" playback of higher quality samples (kinda like how GUS was used with Impulse Tracker on PC and I'm sure several games). A card like this probably wouldn't need to issue an IRQ too often, at least once it's been loaded with sounds (I could be wrong here heh) since, once loaded, the application just needs to tell it what sound to play and optionally at what frequency.

I'm not sure what a driver would look like here. At minimum, programs just need to know which of the 5 I/O spaces the card is configured at. In that context, a 'config.sys' style file might be useful, though particularly with perhaps some Kernel/ROM support so the code that fetches and reads the config doesn't have to be implemented by the program. The program just asks the kernel "at what address is XYZ" - beyond that though I dunno what else there might be needed here.

That said, an I/O card would probably need more driver style support. As in it might be nice to have a library for the card that makes it easier to use in applications. Say if someone wanted to implement a serial interface, a driver could abstract some of the work of the lower level comms so that applications can just worry about the data being sent or received (rather than, say, stop bits or ECC or any of that). In those cases though, being able to jump to the driver code by flipping a ROM page feels like it might be more convenient than having to use the SD card or eat up working RAM? But I would guess writing to the ROM isn't particularly trivial for a typical end user?

Related aside, that also makes me wonder...do we anticipate more than 255 or 65k cards made for the X16? I know "640k is enough for anybody!" but if the cards were limited, they could be given a unique ID which could be part of a card-database file on the SD card? Or maybe that's getting too complicated. It'd just simplify things so the app can ask the kernel: "Does card 123 exist in the system? If so, which I/O space is it at?"

The other thing we talked about was cards that would need to fire IRQs and what the impact of that would be. But I think that might fall on the app side - apps should know what hardware they want to use and so having them setup the IRQ routine for the hardware they care about makes sense? I dunno I'm starting to ramble on here so maybe I'll just leave it here for now.

 

I think my grand takeaway, and again I don't have enough suitable knowledge to really say in fairness, is having a hardware and software standard card makers agree to abide by could also make this easier for everyone. Easier said than done since everyone has an opinion, but I think given the goals of the X16, a solution that is easier to understand over one that was the most efficient, to me, makes sense - but yet again I dunno what that actually means in practical terms ? I do get the feeling it's a bit of the wild west at the moment and so hopefully we see more feedback come about here. I think it's a hugely important topic myself so I'm glad you spent a lot of time to explain a bit of how this all works and hopefully start a good conversation over this.

[Lorin Millsap]

Oh certainly. Things that needs drivers are things that the KERNAL supports. The KERNAL at this moment doesn’t care about sound. So technically a sound card wouldn’t need driver support. That would be a case where each program that uses sound features would need to support such cards.  The types of cards that would require drivers are the types of things where the KERNAL supports those functions. This would include things like RS232/network, storage, keyboard/mouse, etc. BASIC uses the KERNAL routines so anything that interfaces via the KERNAL will automatically work in BASIC.  So a driver is really just a standardized API. Which in this case are just a simple jump table for standardized routines. A driver just hooks into that standard. If there are no existing routines to handle a certain function then you don’t need a driver either, software that can use the hardware will just directly access the hardware.   Sent from my iPhone using Tapatalk

[TheUnknownDad]

I really appreciate bringing this topic together in one new thread.

Also thank you for explicitly describing the hardware environment expansion cards will live in. I hope to see more specs about the expansion ports soon (like pin setup, timing diagrams, electric parameters and later on also size etc...) These specs might already exist somewhere - I am sorry, I didn't search for them yet.

I think we are already developing a common mindset. I support the main idea, a "driver" is "premade code to provide a higher level API" for accessing functions provided by expansion cards. In my opinion, we should have a look on which "hooks" does a driver need towards the system to integrate nicely. I would say, main aspects of these hooks could be "installation", "initialization" and "usage".

Installation is where the system should try to seemlessly integrate the expansion card into the system. If the card complies to a to-be-defined standard, it could be identified as a new card, drivers could be searched for, interactively be requested by a system installation software (my favourite). This aspect splits up into: identification of new cards, identification of (and possibly user support on) configuratin conflicts, getting "drivers" into the boot system and configure specific settings for that card/system setup.

You clearly stated, the selected way so war is dip-switch-like selection. There could be a standard for this, telling people to configure cards in slot 1 to line 1,... This is a guidance not to run into trouble by stting up simple rules. Card developers need to support ALL ids. Again, this is just a convention - and a good one I believe.

Since I hope, the install/config software is all included (no batteries needed...) I would hope, that there is a built-in way to control the boot process if there are errors (possibly unexpected ones) arising. Like "hold F8 during reset will boot in 'no driver loaded' safe mode". Just to bring in one idea how this could be solved.

Convention should also tell developers how to tell the installation program, what driver is needed, which options it has, things like this. 

A convention has the advantage that there could be a reference implentation and automated tests against it. Card developers could then test their implementations before distribution, that rises quality a lot and makes the users happy, too ?

Since things get a litte lengthy here, I'd like to talk about "initialization" and "usage" in the next post.

 

[TheUnknownDad]

So "initialization" is the process of running initial code and integration of other parts of it for future use in a "Terminate-Stay-Resident" like of way. I strongly appreciate a way of identifying that the driver loaded and initialized ok. So I believe a hook to the kernel in form of a return code including a possible error message would be great. The user likes to be informed that the card will not work as expected. Even more he would like to know what did not work out during this boot. It is an essential part of debugging in real world situations...

Initial code is "easy" since it will executed once at boot time. Providing an API to be called on demand or even by interrupts is to be looked at too. Again - a good convention would tell e.g. that the code always has to be relocatable and location is given during initial boot phase. Therefore the kernel should know what the driver needs - which is a convention again, as it does not need to be implemented in code. Might be metadata for the installer, which will do that definition of the memory layout for all stay-resident drivers and give that as options to the driver's init code.

It would be nice to provide a design philosophy to developers which would say: make your device as easy to use as possible, (but not any easier...) - so please provide a BASIC extension which adds high level function calls to access your cards functionality. If that is not possible, provide at least and in any case a "low level" API, namely (relative) entry points to functions, a description for those including how to provide functions arguments. This way the cards functions can be made use of by other users and developers at high or low level. Wait, did we say relocatable code? ok, so how do programs find that location? aahh, there is a kernal call for memory addresses of expansion card drivers, right? great!

I hope I could make my point clear without any offense. I believe that few functions are needed and a good convention with a test suite for developers.

Result would be a real product, still fully flexible but more compatible. Why bother to integrate more than one expansion port if no rules are set how to operate more than one card ? 

I am eager to hear/read your thoughts on this.

 

P.S. You wrote, that you had a chat yesterday - is there a public chat like a discord channel or was that some internal chat only? If it was public, would you mind telling me how to join that? Thanks!     

[BruceMcF]

The argument against the  config files being Basic is Basic doesn't handle calling subprograms very well.

Now, it can be gimmicked with something along the lines of

10 IF F1 THEN 20:F1=1:LOAD"F1.BAS":RUN

20 ...

... and then the subprogram loads the main program.

However, this only works if the main program is bigger than the subprogram, otherwise the subprogram trashes the variable space of the main program.

One idea is to fix this with a special load keyword for the main config program that loads it AS IF it was a fixed size, even if it is smaller, so it is known what size subprograms can be without trashing the main program. If the filename is fixed, like CONFIG.BAS, then the main program load keyword doesn't need to include the filename, and it can simply be the last basic command in any subprogram.

So, for example, suppose that CONFIG is the keyword. It loads CONFIG.BAS and when the load is finished sets the program end vector to $7FFF. Now subprograms that are smaller than 30K can be run without trashing the main program.

This makes the High RAM and end of Low RAM memory management orthogonal to the configuration file system.

I also like the idea of a way for a driver to check whether it is already loaded. One idea is to have a defined High RAM segment that contains user readable configuration data like a HighRAM segment BAM, which could also contain a linked list of names of loaded drivers, so an installation program can check whether it's not yet loaded, or perhaps it's been loaded but taken out of the API chain.

 

 

[TheUnknownDad]

Parameters can be placed in seperate files that could be of SEQ type. All we need to know is, which parameters to put there and which size/format they should have. The driver could then still be a single BASIC or assembler program.

However, the problem with subprograms would not be solved by this idea.

I think it would be a good idea to make a sketch a workflow of the whole process in which we could "top down" detail each step further. At the end there should be a full blown workflow of our suggested activities, each part has to do when following that convention. After having identified, which parts need additional software or software changes, we could start working on these and finalize a RC for the expansion convention. Sketching the workflow will also help others to understand, what our intent is and get a better understanding of the overall process. Would that be ok for you? If so, any suggestions where such a sketch could be graphed online, so we all could interactively change it? Would Google be a candidate?

While reading a note from another user, it came to my mind, that there could be a non DMA, non overlapping solution for the "include a driver in the card" thing:

On card installation process, when running the installation software, expansion cards could be in a "uninitialized phase" and provide a standard interface for information on the card itself, including an interface to get the built-in driver in a byte-by-byte interface, just like data transfer to VERA works, but in the other direction. The install software can get the driver this way and put it on the SD card. Any comments on that idea?

I believe, there should be this two phase standardized talking to expansion cards in any way. By using only few I/O RAM spaces for that communication it would also be possible to add this without really implementing two phases on the card, since it could be like first 8 bytes are init phase standard use, leaving 24 bytes for free use without extra cost of implementing 2 phases on the card. If it needed all 32 bytes, fine - the driver should set it to phase 2 mode after finalizing initialization. Again, any comments?

[paulscottrobson]

Is DMA Access going to be a thing with peripherals, specifically Vera ?