Commander X16

34 minutes ago, TomXP411 said:

I'm really hoping the source code gets released, because I really want to see Commander on the MiSTer FPGA platform. All of the other components in Commander already exist as open source FPGA code; it's just VERA that is unique, at this point ... Then you don't really understand the motivation behind the project. Why do you think the emulator, the firmware, and all of the documentation are already released under open source licenses?

You both have some good points, but just to repeat what has been said before, and is posted in the rules at the top of the main forum page:

Quote

No clone machines: The X16 code is years of combined work & includes protected Commodore I.P. that was hard to secure. Only our team have permission to use it in this way. The unreleased X16 is not open source, yet.

So as of now, we're not discussing open sourcing or ports to other machines. Our machine isn't even out yet. So let's save this discussion for a later time eh? ??

I think it should also be reiterated that there is a difference between open software and system development and open source licensing. The fact is that there is no open source license for anything in the X16 project. The team has merely been open with their process and making their software repo public to better engage the community. At any point, those repos can be taken private and further development done in confidentiality. There is no legal means for people to do anything with the ROM code or emulator other than use them as intended.

3 hours ago, SlithyMatt said:

The fact is that there is no open source license for anything in the X16 project

That's not true,

 

• 
  All new code, and additions to legacy code: ©2020 Michael Steil, www.pagetable.com; 2-clause BSD license
  


• 
  FAT32 and SD card drivers: ©2018 Frank van den Hoef; 2-clause BSD license
  


• 
  kernal/open-roms: ©2019 Paul Gardner-Stephen, 2019; GPLv3 license
  


• 
  Emulator, Copyright (c) 2019 Michael Steil <mist64@mac.com>, www.pagetable.com. All rights reserved. License: 2-clause BSD
  


• 
  
  Commander X16 Documentation (CC BY-SA)
  
  

@lamb-duh I stand corrected... partially. I should have said that the X16 system and ROM are not holistically open-sourced licensed, but do contain open source components. I'm not sure of the legality of the emulator's BSD license, as it is emulating a proprietary system. But, as it was made with permission of David and the dev team, there is some wiggle room there. It does not mean, however, that you can go ahead and reverse engineer an X16-on-a-chip using the emulator as a guide, or using the pictures of the motherboard PCB.

23 minutes ago, SlithyMatt said:

@lamb-duh I stand corrected... partially. I should have said that the X16 system and ROM are not holistically open-sourced licensed, but do contain open source components. I'm not sure of the legality of the emulator's BSD license, as it is emulating a proprietary system. But, as it was made with permission of David and the dev team, there is some wiggle room there. It does not mean, however, that you can go ahead and reverse engineer an X16-on-a-chip using the emulator as a guide, or using the pictures of the motherboard PCB.

You're right, the hardware is not open and a big part of the core software is not open. There's decades of case law that lays out exactly how to legally clone a closed platform. If someone wants to make a clone of the machine and sell it, they will. Saying "no clones" isn't going to solve the issue. Emulation is legal, by the way.

41 minutes ago, lamb-duh said:

You're right, the hardware is not open and a big part of the core software is not open. There's decades of case law that lays out exactly how to legally clone a closed platform. If someone wants to make a clone of the machine and sell it, they will. Saying "no clones" isn't going to solve the issue. Emulation is legal, by the way.

Thankfully we don't have to delve too deeply into that subject, because the emulator is being made available, for free, and folks are welcome to contribute to it through bugfixes and improvements. I've personally had a lot of pull requests accepted by now, too, so I'm definitely speaking from personal experience contributing to the emulator.

Actually, I've been thinking about starting a subject to discuss what technical shortcomings might be addressed in the emulator... for instance, the VERA is probably no longer faithfully reproducing NTSC graphics. I don't think anyone has tested line IRQs on NTSC, either.

We also need some unit testing for the emulator, either prgs so we can capture the output to compare between real hardware and the emulator, or a unit testing framework in the emulator so we can run tests to compare output state of the machine with known correct examples. (Unit testing is something that @Michael Steil has directly brought up a couple of times, especially recently.)

...and this is where I blather on for 4 paragraphs about optimizing the emulator, so feel free to stop reading if you're not interested...

On the performance side, I know there's interest from multiple folks to try and run the emulator on RPis. I have a brand-new RPi4, and it runs the emulator at about 40% normal speed, or 45% when the new "warp" mode is enabled (it's not in the r37 release, but it's in the github repo and will be in r38). 40-45% is a long ways from running at full speed on an RPi.

Besides the VERA, which will likely always remain a pain point, it looks like there may be some gain to be had from optimizing the PS/2 emulation (comments in the code actually directly point out a couple of places where improvements could be made). Profiling with some specialized tools has also suggested that the basic CPU emulation could be sped up by around 10% if folks found ways to remove branching from various opcodes, such as when we branch between ordinary binary arithmetic and BCD arithmetic. In the the case of BCD, specifically, perhaps we could split the instructions into binary versions and BCD versions, and then alter the function call table whenever we run SED and CLD instructions, so that branching within the instruction itself is unnecessary and we only pay for the cost of flipping contexts. Or, since the memory to emulate an 8-bit machine is hardly a concern even on a meager RPi, we could just keep two instances of the entire instruction table for that case, and choose one as the "current" table. Or I'm sure there are other approaches with varying pros and cons, and I'm sure we'll discover other pain points as we play whack-a-mole with hotspots.

And when it comes to the VERA, there's a lot of work it could choose to do once, and then rely on cached data to speed up the final draw process, and refactoring for that would probably open up more opportunities for multithreading (actually, I've been working in this direction, and there are lots of opportunities for multithreading when we attempt to do most of the work up-front instead of re-performing work every single line). Also, the VERA emulation makes almost no use of SDL2 functions to try and accelerate any of its tasks, so that's another avenue possibly worth exploring. And then there's sprites. Sprites are hard, and there's no two ways about that, especially if we want to emulate the VERA's work limitations with sprites.

But overall, the VERA is in fairly good shape for now... it's expensive, but even turning it off (effectively what the new "warp" mode all but does) only gets us from 40% to 45%. I mean, sprites are still a major pain point even in warp mode, but that's because warp mode still calculates sprite collisions. But it's not all VERA, all the time, at least not anymore, thanks to the contributions of at least several people, and hopefully more in the future.

2 hours ago, SlithyMatt said:

I'm not sure of the legality of the emulator's BSD license, as it is emulating a proprietary system.

In the abstract, it's not an issue: emulators and clones are perfectly legal in the US, so long as they don't include patented hardware or Copyrighted software without a license. 

Up until the Cloanto deal, the team was perfectly happy to let us discuss alternate implementations, and several popped up. I understand the reasons; Cloanto doesn't want Commodore software distributed without a license. So I'm happy to wait until some sort of licensing deal for alternate implementations is available. 

I will soon be building a 6502 computer, because I want to test some VIA code. Since I'm unlikely to get a Commander prototype, I'll have to build one of the Apple I clones, instead. 

 

 

Sprite collision calculations seem something especially suited for multi-threading.

3 minutes ago, Kilian Hekhuis said:

Sprite collision calculations seem something especially suited for multi-threading.

There are hard problems.

First, sprites are allowed to move between committing one scanline and the next, so you can't just draw the whole screen once per frame, you have to spin through them every line.

Second, not all sprites are necessarily drawn on a line. You have to know how many work units that line of that sprite will cost, in order to know whether future sprites on that line even have a chance to be drawn. This is really the rub, because you can't just run through all 128 sprites in parallel and try to rely on atomic operations (assuming said atomic operations are portable to other processor architectures) to ultimately converge on a collision mask. Maybe there's a radically different approach than what's already there, which could be multithreaded to get this job done super-fast, but I've been slowly noodling away on it for a while and nothing's come to mind so far.

I can imagine having to almost duplicate the sprite rendering engine (if you could call it that) for the sprite collision detection, but if that allows processing in a seperate thread, I'm all for it :).

That said, I'm well aware of the difficulties of creating a fast emulator. I'm currently developing my own retro home computer using FPGAs, and the biggest challenge I had in creating the emulator wasn't the synchronisation of write events (the CPU changing registers at some point in time), but read events (the CPU postponing a write until some value was read, e.g. syncing to horizontal retrace). Write events can be put in an event queue, and syncing that with the screen rendering can give you a pixel-perfect and somewhat fast emulation, but the reads are deadly... In the end I decided not to solve it, and rely on the software playing nice and using the hsync interrupt instead of busy-loop waiting.