User:Eekee

Update: I've been slowly improving at Forth, but now I'm finding Plain English Programming a huge leap forward. It completely sidesteps my problems reading code, and so I have a much larger set of examples to learn from than with any other practical systems language. It helps that most definitions are very small, a point encouraged by the absence of nested loops or conditions. Forth was also rather good for me, but required other skills which were only coming to me very slowly. Even in the simplest definitions, tracking stack order is still sometimes a challenge. Organizing data in memory is something I'm not as good at as I thought I was. Plain English has high-level types. It has conventional local variables too, but gives them generic names based on their type. I like that. Lisp was my second choice, but Plain English is actually a little closer to the machine. I like that too.

I'm not sure I'll do any OS development in Plain English, but some OS dev concepts are applicable to the plans I have for it. The IDE (which doubles as a sample program) rather resembles an operating system in use, being permanently full-screen with its own way of working, but that's more a point of curiosity than anything technical.

I still might make a Forth interpreter as a BIOS or UEFI application. Plain English would need a bit more work, being 32-bit only at present.

---

Hello! I'm the offspring of James T. Klik and Lino Commando! I'm joking, but I am slowly developing a system with mixed REPL and mouse input. It's not necessarily going to run on real hardware or even Qemu, but I do rather dream of it doing so. Similarly, I like to flirt with Alta Lang but in practice I'm 'just' developing on Forth, with plans to build an extensive library. My dream is for it to run under operating systems and directly on hardware, a possibility achieved by Inferno and Oberon. Also like Inferno and its parent OS, Plan 9, I plan to make heavy use of IPC. Unlike Inferno and Plan 9, it will not be "just plain text", structured data will be the norm. I'm not entirely sure why I'm targetting hardware, but there's a certain freedom in it and it does rather satisfy a very long-standing ambition.

This system is Kaph. It was a huge project, a whole panoply of high-level goals, which rather put me off for a while, so I came up with some other projects and listed them below. Now I see I can begin by implementing a Forth, build a partial version of Kaph on top of that, and see what changes and additions need to be made.

More about me

My levels of experience are quite atypical due to my health holding me back from accomplishing much but not always holding me back from observation. I've had decades to absorb concepts and see what does and doesn't work, but little practice. My health is not a fun subject, so let's leave it at that. I'm getting a bit done now, and I'm finding the process of development to be a lot of fun in itself.

I have had some practice. For instance, some time ago, I tried to be a Real Man, but then I learned to relax. I am definitely not Barry Pi, I'll take a consistent architecture, thank you. The existence of Andy Microbaum is tangential to my universe, much like it is to Plan 9. I don't need to say "I love these archetypes," do I? :)

One day, Uriel met a man who argued convincingly that Plan 9 is a microkernel.

On another day, Uriel met a man who argued convincingly that Plan 9 is a macrokernel.

Uriel was enlightened.

I have dabbled in CPU design a little. I'm obviously tempted to pursue this further, but I seem to have even more to learn about this than OS design, and that's just for a "simple" CPU; no MMU. Trying to design 8-bit CPUs alerted me to some surprising complexities. Still, when I look at Dawn, reviewed below, I start thinking of it again. If you want to design CPUs complete with their own instruction sets, I recommend starting with a larger word size than 8 bits.

Ultimately, I'm sure it would be much more fun to develop for a made-up architecture, even if only in emulation, than for any hardware containing such horrors as ACPI. I'm told USB is very unfriendly to operating system development too (although I'm starting to think it depends on your ideas about how things should be structured). Another possibility is developing for emulated older hardware, which in turn can and in some cases has been reimplemented in FPGA. The more time goes by, the more possibilities open up in hardware and the more lines blur. I should remember to look into RISC-V. This all reminds me, I wonder if the GNU universal peripheral bus is anywhere near ready, or indeed if it's sensibly designed.

Kaph Project and Concepts

My plans for Kaph include several distinct concepts which are unusual in themselves or unusual at the OS level.

• An unusual (but not entirely unique) UI design with REPL and modifiable GUI in close association -- Inspired by colorForth and the Sam text editor
• Execute or "plumb" any text, where the "plumber" is a very powerful spin on file association -- Inspired by Oberon via the Acme text editor on Plan 9
• Text-heavy GUI where any text is selectable and much or all text is editable -- To empower the plumber, assist the REPL, increase the power of the GUI (see previous point), and because forcing the user to copy-type is very poor
• A single language embracing domain-specific languages -- Inspired by my difficulties with Plan 9's maze of twisty little languages, none quite alike
• Universal data interchange structures and mechanisms intended to make pipe filtering quicker and easier, and to ease the process of manipulating data with different DSLs
• Modification of in-memory data by transient programs -- Using Acme with Unix-like Plan 9 commands showed me how practical this way of working is. Wanting to use it for images too was a seed idea for Kaph. Some Forth block editors are essentially this; the text's existence in memory is independent of the lifetime of the commands which modify it.
• Unified device interface; perhaps not "everything is a file", but reducing "everything" to a very small number of categories with the help of the universal data interchange structures -- Inspired first by Atari 8-bit home computers, then Unix and Plan 9, but I see a need to avoid oversimplification
• Scalable UI is not quite so uncommon in this decade, but there are still areas where improvement is clearly possible
• Increased availability of undo, especially undoing file changes at the OS level -- Sam & Acme taught me that undo solves the problem of running extremely powerful text processing commands without thinking about them first. I don't think I can make this as system-wide as a certain mainframe OS, but I would at least like it in the filesystem.

UI and window system

Acme's facility of executing or plumbing any text anywhere while almost all the text is editable adds up to an extremely powerful system. I would like to capture this power without requiring a 3-button mouse. If I can make it work on touchscreens, I'll be very happy indeed.

On the other hand, I don't want to imitate Acme's dynamic window tiling. Nor do I want to allow programs excessive control over their windows. I've been happy with simple, static window tiling, and this is what I want to implement for Kaph. I intend to have a single REPL window serving whichever window is current, but I have yet to work out exactly how it will integrate.

Under the surface, a big part of Acme's power comes from scripting it. The ability to execute anything means anything can be a script, whether global, specific to the editor, a script in a directory (very powerful), or a line or two in a plain-text file. Such scripts can communicate with Acme, either by being run under Acme's Edit command or by using the virtual files Acme serves.

The plumber too is programmable. In Plan 9, it is designed to send messages to ports, but I plan to simply have it execute scripts. The scripts can send messages to programs if desired. This implies many programs will be able to receive relatively arbitrary messages, much like Amiga programs with their AREXX ports. (I just thought, this offers a means of implementing async i/o: start a command which will send a message when the i/o is ready. If the commands and message-passing are as lightweight as I hope, this may be reasonable.)

Language

[discuss forth; mention lisp; mention making forth safer; start to consider pep]

Data interchange structures

In Plan 9 and Unix, I got very tired of addressing fields by number or trying to make regexps to match the right part. For example, try picking dates out of ls -l output. It's slightly easier in Plan 9 because the output is fixed; it's not internationalized and doesn't change depending on whether the output is a terminal or pipe, (the latter is sometimes disgusting!) but 'slightly easier' is still far from easy. The fields have names, why can't I address them by name? In Kaph, lists of files, processes, or other things will be represented as data structures with named and typed fields. [to do: more detail]

~~page update WIP~~

old:

A more "meta" thing I learned from Sam Acme and Plan 9 is the power of having a few simple but powerful interfaces with a powerful command language behind them, rather than many discrete applications. Plan 9 isn't actually the best in this regard, having many disparate languages which take a bit of thought to interface together, and the easy way is often the fragile way. This is why Kaph will have the one language, but this is not to say it will exclude DSLs. (DSL = Domain Specific Language, a small language for a specific task.)

Forth is perhaps the ideal language for integrating DSLs into itself. In Charles Moore's wording, it was a 'compiler' before it was a language. He came up with this compiler system and used it to implement "problem oriented languages" before developing a full language with it. See Programming a Problem-Oriented Language.

Lisp is also good for DSLs. Both Forth and Lisp are especially good because they mean you don't have to write a parser for each DSL. You just use the existing parser, only spreading out the syntax of your DSL a little bit. I even prefer this spreading; I find printf and regexps too terse to read, and I hate having DSLs stuffed into strings where normal code formatting practice breaks down.

I have been tempted to use some variety of Lisp instead of Forth, but I'm not keen on its basic principles. I would rather use a language closer to the hardware and find a safe way of working with it.

Minor Projects

Various Native Forths

Because Forths are relatively easy to make and the results are quite satisfying. Also, it's a form of practice for some aspects of OS development. I may write some up as barebones tutorials.

Progress and plans:

• I have a direct-threaded real-mode x86 Forth in development. It might be working already, but I felt it important to concentrate on personal development instead.
• I have some embedded ARM hardware; the obvious next target.

'Threaded' here is Forth interpreter design jargon, not a word for one form of multitasking. It's because the execution flow jumps between routines in the manner of a literal thread in a complex weave or seam. I initially intended to immitate the double-indirect threaded Jonesforth, but all that indirection was too much for my brain. Besides, double-indirect threading was invented to save memory and was apparently faster on some hardware, but stores addresses as data. These days, I think I'd rather save the data cache for actual data. Direct-threaded Forths are simpler, and Pygmy Forth for DOS shows they are perfectly capable of being fast and tiny even on an 8088.

There are many public domain Forths; plenty of code to use if desired. This of course raises the possibility of just porting one instead of developing from scratch. However, developing Forth programs is greatly improving my programming skills.

A8.86

When I was 13 or 14 years old, the Atari 8-bit Home Computer Operating System Manual showed how many peripherals of very different types and needs could be operated through a unified, consistent interface. It blew my mind! When I first got a MS-DOS reference book some years later, I was repulsed by how ad-hoc and junky the interfaces were by comparison. Naturally, I wanted to make an OS imitating Atari's design, but I didn't have the programming or organizational skills in the 90s. When I got Linux years later I found some clean Unix interfaces, but many obstructively non-uniform interfaces too. I was also astonished to find how primitive and generally rough Unix terminal interfaces are. Atari provided a full-screen text editor for all-purpose input starting in 1978! Plan 9 went some way toward fixing all this, but was still disappointing in some ways as outlined under Kaph above. So, if I had time, I'd still like to produce an OS imitating Atari's. :) However, most of the features I want have ended up as part of Kaph.

I might still do something along these lines as a sort of Kaph-lite. It would have to have some interesting graphics hackery because I'm imitating Atari. For you Amiga fans, Amiga Inc. was working with Atari before they approached Commodore. I don't know how closely they worked together, but the graphics features some find so amazing in the Amiga are entirely typical of Atari's way of thinking. ;) They're obvious next steps from the graphics systems of the 8-bit Ataris. I don't blame Amiga Inc for ditching Atari; Atari's boss at the time was a horrible person to work with and Commodore offered them literally 5 times more money.

You know, "interesting graphics hackery" suggests a hardware project... I'm thinking "A8.ARM" hardware running Kaph. That would be great, if not too much of a challenge! :D But speaking of challenges and hardware, I've had thoughts of designing my own CPU...

Suggestions

Remote Shinies

If you want to be James T. Klik, may I recommend becoming Nick Stacky first? Using other operating systems' graphics drivers is great! Of course, you will have to design a networked window system or interface to X. Don't try to use Wayland over X, basic input breaks down over the network! VNC is also a bit rubbish in the input stakes. X may be tough, but it gets it right. Plan 9 also gets it right, but doesn't have drivers so it's more of an example you may wish to imitate. Plan 9's windowing is much simpler than X, but note that you'll have to understand its filesystem semantics to understand how its window system is exported correctly. In places, it takes a genius to understand Plan 9's simplicity! ;)

Reviews

Note to self: try more OSs! I had plans for this section, but I always seem to be doing something else. Occasionally, the "something else" is even OS coding. >.>

FreeDOS

I dip into FreeDOS every now and then, especially to code in 16-bit assembly language, but almost everything is unstable. It's just-about usable on my Thinkpad x61 (2007), was hardly usable on Qemu years ago, and seems even worse on a more recent version of Qemu. Even on the Thinkpad, many editors will crash or develop unpredictable bugs if you touch the thumbstick while pressing a key.

I use software from the 3 released FreeDOS ISOs. The best text editors have severe interrupt/keyboard issues under Qemu, although they're not so bad on real hardware. They're all very different from each other, most having as little relation to each other as Vim has to Notepad. They also tend to have peculiar design choices -- I have a file with notes on each editor which makes me alternately laugh, cry, and yell WHY? ;) I use SetEdit (despite instability) because it saves its session when I quit to run Nasm, and I use very simple batch scripts to run Nasm. I prefer quitting the editor to starting a subshell because the subshells of many editors don't work in FreeDOS, (especially not in Qemu,) and they are often cumbersome to invoke anyway.

There are several GUIs for DOS. 2 or 3 of them appear to be virtually OS projects in the way Windows 1-3 were OSs built on DOS. They're unstable. OpenGEM is stable, but preserves some of the the worst clunkiness of 80s GUI, including clearing the selection when you scroll. (Atari fixed that, but the fix never reached OpenGEM.)

I stuck with the shell, using COMMAND.COM-compatible FreeCOM until I found it difficult to grep for a regexp containing a single double-quote character. I had to add a double-quote to the file list and put up with the "file not found" error, otherwise FreeCOM would ignore the redirection character on the same line. Now I use 4DOS which has a proper means of quoting arguments.

4DOS also allows setting the default text colours, reducing the problem I have with bright-for-bold displays. However, the problem returns as soon as I pipe text into a pager, or run some unconfigurable programs or run an editor whose configuration doesn't work.

I've had a little fun with assembly-language coding and some Forth and BASIC interpreters, but I don't really enjoy FreeDOS, or any DOS. There are some nice games, but I'd rather game on my tablet. And the one Forth interpreter I want to keep using, I run under DOSBox.

Nano-reviews

• 9front I used full-time for about 10 years! In a sense, it was my primary OS. I only used Linux for web browsing, 3D virtual worlds, and occasionally OpenTTD or some other game. Everything else I did on 9front. However, there were some frustrations. I didn't have enough free creative energy to overcome the lack of configurability. Many of the little languages were uncomfortable, and so were the differences between them. If you're comfortable with such languages and with deeply sophisticated C, it's a very special system! So is its parent, Plan 9 from Bell Labs.
• KolibriOS is a bit of a favourite because it was an early find which is impressively complete and capable, and because it has simple APIs. However, its GUI is locked in the past with tiny fonts I can no longer comfortably read. I have endured many traditional GUIs since the 1980s so I can say KolibriOS's GUIs are better than average (apart from the size); stable and less clunky than some, but it's not up to the UI standards of the present era.
• MenuetOS gets a mention because it's KolibriOS's parent. I believe the APIs are also simple even for assembly-language programming, as are Kolibri's. It's certainly capable of rendering larger, smoother text. However, it has very few user programs by comparison, perhaps it because it switched to closed-source almost at the height of open-source enthusiasm.