awk or perl-like userland scripting is needed

If you have feature requests, this is the place to post them. Please note your idea may already be something we have already discussed and decided against, or something we are working on privately, and we cannot be held responsible for any similarities in such instance. Whilst we cannot respond to every suggestion, your idea will be read and responded to where possible. Thank you for your input!
Edmond D
Posts: 467
Joined: Thu Aug 19, 2021 1:42 am

Re: awk or perl-like userland scripting is needed

Post by Edmond D »

Thanks Bruce for the example and explanation. I can see its use for the developer. I wonder how much development would be done needing scripting on the X16 itself once the hardware is released? Given the amount of software already developed using cross-compiling tools on off platform systems leads me to that question. I do see value in the experience of attempting to work completely on the X16 in a stand-alone environment (that's so Eighties! :) ) Perhaps others who have developed inside the emulator (Basic, Assembler, Forth) might weigh in, as those are the only languages I know about that I believe have an on platform development environment. If C is in that mix, apologies to those programmers, or any other language I've missed.

As for us near mortal minions (end users?), I don't see a compelling case for a scripting language yet. I'm open to hearing others suggestions why the X16 needs one. Consider this a call for education, not an assumption that scripting isn't desirable. I'm all for making the X16 meet the community's needs, no matter how wide the feature request they are....
orangeman96
Posts: 4
Joined: Tue Mar 14, 2023 3:09 am

Re: awk or perl-like userland scripting is needed

Post by orangeman96 »

I'm loving the discussion here...

My experience programming in MS Basic (on the VIC-20 & Apple ][ ) as an Elementary & Middle School student, coupled with UCSD PASCAL in High School, enabled me (after a 15-year hiatus) to reform a parallel analysis system by leveraging my knowledge to code to leverage RAM versus burdening IO by relying on write-read to storage as a "simpler" ("lazy," I contend) approach.

Doing so resulted in scripts (née "programs") that had taken dozens of minutes (some not finishing before 'cron' kicked off the next iteration) to less than five minutes (many less than 90-seconds), "breathing 'new life'" into the system by freeing up IO for "non-cron" processing.

My take is that "scripts" ("batch programs" to the Microsoft-trained "coder") should be secondary, and used sparingly, while programming to leverage RAM should (once again) reign supreme.
orangeman96
BruceRMcF
Posts: 223
Joined: Sat Jan 07, 2023 10:33 pm

Re: awk or perl-like userland scripting is needed

Post by BruceRMcF »

Edmond D wrote: Fri Mar 24, 2023 7:54 am Thanks Bruce for the example and explanation. I can see its use for the developer. I wonder how much development would be done needing scripting on the X16 itself once the hardware is released? Given the amount of software already developed using cross-compiling tools on off platform systems leads me to that question. I do see value in the experience of attempting to work completely on the X16 in a stand-alone environment (that's so Eighties! :) ) Perhaps others who have developed inside the emulator (Basic, Assembler, Forth) might weigh in, as those are the only languages I know about that I believe have an on platform development environment. If C is in that mix, apologies to those programmers, or any other language I've missed.

As for us near mortal minions (end users?), I don't see a compelling case for a scripting language yet. I'm open to hearing others suggestions why the X16 needs one. Consider this a call for education, not an assumption that scripting isn't desirable. I'm all for making the X16 meet the community's needs, no matter how wide the feature request they are....
As far as how much hosted vs cross development is going to happen -- most of the hosted development that takes place will take place when people have the hardware to work on. Whether it's 20% or 5% or 0.2% of the development work is not something I worry about. It's how I'm going to develop on it if I get my hands on one, so for me the cross development is more a necessary evil until the hardware is in my hands and more full featured hosted development options have been developed.

As far as the use case, does it have to be a compelling case? Is "handy" or "convenient" good enough?

It seems like people are talking about scripting from a "Unix as a server operating system" perspective, rather than the much broader range of uses for scripts we had back before we had the mountains of user libraries and the millions of cycles to burn that are the foundation for languages like Python.

A scripting language like awk is a convenience that makes it easier to accomplish something. In general use, it is used to automate something that would be tedious to do manually but also without the higher investment in programming time and effort required by a more general purpose language.

So, for a specific example, suppose that I have a block Forth file from back in the day that I want to port to xForth and I want it to load as an INCLUDE text file. Assuming for the sake of the example that I have fixed the CREATE ... DOES> bug and xForth actually works as a Forth, to be sure I would write a Forth word to convert from the (no return character, fixed 64 character lines, 16 lines per block) BLOCK format into a text file, replacing all of the space at the end of the block line with a carriage return.

To assist with the port, I went the text file to have an extra carriage return at the end of each block, and an extra comment line at the start of each block that says, eg, "\ BLOCK 34".

That's something where it would be convenient to have a scripting language to do that, since it's just a couple of lines in some "small AWK" language.

Whatever assembler hosted on the X16 that I use, cross development assemblers are likely to be more forgiving in terms of the various of different layouts that they accept than a hosted assembler -- because of the "resting on a mountain of libraries" nature of the modern cross-development assemblers -- so when I download assembly files that people have worked on in a cross-development assembler, I may have to go through the file making sure that it complies with the less forgiving layout requirements of the hosted assembler.

To be sure we can write the lint program in Small C or Pascal or Forth or Basic, but it's quicker and easier and easier to customize to a particular case -- eg, one common cross-development assembler *requires* something that some hosted assembler *forbids* -- and easier to test and overall likely to be less error prone to have a lint script to run through a scripting language.

Now, as far as a feature request, I don't think it has to be in ROM, so it's more an encouragement that if someone wants to work on it, if they come up with a useful "small AWK" (or whatever), the effort won't be in vain.
c64c46c
Posts: 9
Joined: Thu Mar 16, 2023 3:09 am

Re: awk or perl-like userland scripting is needed

Post by c64c46c »

Thank you for the serious considerations here.
TomXP411 wrote: Tue Mar 21, 2023 8:40 am Does this mean you are volunteering? :)
Depends on "for what". I'm super excited about this programming environment and am more than happy to contribute where I am able. My expertise is not in 8 or 16 bit programming, but I am a programmer of many years and strongly motivated by this platform (and my childhood machines, VIC-20/C64). I have something of a background in computer science and perl/awk. I plan on buying an x16 board when they become generally available and am currently learning to 8 bit program on my childhood C64 (and trying to resurrect my old VIC-20).

I am not a fan of emulators when I can touch the machine, but understand the current stage of things and have been meaning to checkout the x16 emulator so I can learn more about the programming and the dev workflow it enables.

So I DO volunteer to be around and learn more about this very exciting programming environment. Other than that, I will have to find out where I fit still. An awk or perl-like programming environment would be that niche, but understanding the need to blaze a trail for certain things we take for granted in the more "modern" environments, I am not afraid to learn to wield a machete or two.

Cheers!
TomXP411
Posts: 1745
Joined: Tue May 19, 2020 8:49 pm

Re: awk or perl-like userland scripting is needed

Post by TomXP411 »

I'm glad to see your interest.

So far, ecosystem development for the X16 has been ad-hoc. "See a need, fill a need," is very much the motto of the day. There are a few projects in play, such as the EPROM programmer, at least 3 language projects, and several games. These are all people who just decided to start making something.

However, that's not to say that everyone's favorite interpreter or compiler is going to make it into the official ROMs. There's only so much room, and there are a few things that The 8-Bit Guy has prioritized. So consider how you'd implement such a thing. For example, using the 8K ROM banks for storing scripts and variables and using the 38K of transient program area for the interpreter.
Edmond D
Posts: 467
Joined: Thu Aug 19, 2021 1:42 am

Re: awk or perl-like userland scripting is needed

Post by Edmond D »

BruceRMcF wrote: Fri Mar 24, 2023 4:28 pm
Edmond D wrote: Fri Mar 24, 2023 7:54 am

As for us near mortal minions (end users?), I don't see a compelling case for a scripting language yet. I'm open to hearing others suggestions why the X16 needs one. Consider this a call for education, not an assumption that scripting isn't desirable. I'm all for making the X16 meet the community's needs, no matter how wide the feature request they are....
As far as the use case, does it have to be a compelling case? Is "handy" or "convenient" good enough?
Sorry for the delay, but thanks for your elaboration - most appreciated. Handy/convenient works for me. Given the direction of the system to be capable stand alone development platform, I can see the need. If it was a "cartridge" based scripting environment (much like the Vic-20's programmer's aid cartridge) then those who want it as a handy tool would be able to use it. If the scripting language was compiled to be relocatable that would be cool for those who don't want to invest in hardware, or need to move it out of the way of their code. I can see the already developed text editor being a useful environment for writing the script, so that part might be done. It's just the scripting engine that supposedly needs development.

Unfortunately, I don than the implementation skillset to help in that endeavour. I do have a testing/QA background to offer, but that requires there to be some code first. having been around motivated developers long enough I forecast it's a "someday" reality.
Post Reply