[Debian-lego-team] BrickOS-Bibo Project Updates

[Matthew Sheets]

Hello Nicolas,

> This is nice!

Thank you!

> I prefer using a Makefile, but I suppose some people will like it.

Yes, I can understand that.  :-)  From other feedback, I anticipate the script will generally appeal more to the beginner or casual tinkerer lacking prior familiarity with Makefiles.  The number additional features above the standard firmware even when using NQC, like being able to program all four keys on the keypad with fairly complete LCD control while also having extra capabilities like DCC available, seem to be some of the particular draws to this platform despite the increased complexity.

> I will have a problem with brickos and brickos-bibo, if I keep a package
> for both, there will be a collision on the command names. Not sure how
> to handle it yet. I think bibo is sufficiently different that having
> both may be valuable, but maybe I am wrong.

Short Version: After adding support for the collective versioning of include files, brickOS v0.9.0 should now be able to be handled as simply another kernel version (e.g. named "legacy-0.9.0") within the newer ecosystem from the brickOS-bibo repository.
* brickOS v0.9.0 would need to be repackaged to follow the new multi-kernel conventions and also have its *.lds file update to provide BASE1 and BASE2 (can be done manually or via a patch file)
* Both header files and library are now collectively versioned, with those for brickOS v0.9.0 being "v0"
* Code in the brickOS-bibo repository will be bumped to a version 1.*.*, with the header files collectively versioned as "v1"
* The "headers" targets have been removed as a prerequisite of host, so the high-level independent targets are now host, headers, kernel, and demo.  Perhaps packages could potentially be split out this way, too?
   - Host:  A single package for this, built from brickOS-bibo
   - Headers:  Different package versions for the different header version groups (e.g. v0, v1)
   - Kernel:  Separate packages for different kernel configurations plus legacy-0.9.0, or maybe just a "default kernel" package, with version 0.9.0 being legacy and version 1.* being bibo?
   - Demo:  Haven't gotten to this yet, as the demo targets are not as integral to core functionality, but ,ight need to do something similar with demo like was done with the headers…

Testing:
* Manually created a directory structure and renamed files to setup brickOS v0.9.0 as a multi-kernel instance, using files extracted from the brickos *.deb package
* Updated the legacy-0.9.0.lds file to include BASE1 and BASE2 (BASE1 = app start and BASE2 = BASE1 + 0x0210)
* Created a legacy-0.9.0.lxMakefile.conf with the line "VERSION_GROUP = v0"
* Built the legacy-0.9.0 demo C program "helloworld" by using from the brickOS-bibo project the new "makelx" script file with the new Makefiles
* Deployed the legacy-0.9.0 firmware using the latest "firmdl" from the brickOS-bibo project
* Downloaded the "helloworld" program built for the legacy-0.9.0 firmware using the latest "dll" from the brickOS-bibo project
* Successfully ran the "helloworld" program on the physical RCX
* For good measure, I also tried this over the USB IR tower on a custom-built WSL kernel that has the LEGO USB tower drivers enabled that is running under a Windows 10 64-bit installation—and it worked  :-)
* NOTE: While legacy-0.9.0 *.coff or *.bin files are not required, at least the *.coff build of the firmware is needed in order to enable the BrickOS integrations in the emulator, such as program side-loading

The high-level structure of this manually-created "legacy-0.9.0" multi-kernel instance with headers v0 used in this testing was as follows:
./h8300-lego-brickos-coff/boot/legacy-0.9.0/legacy-0.9.0.[lds,bin,coff,srec,lxMakeconf]
./h8300-lego-brickos-coff/include/brickos/v0/c++/*.H
./h8300-lego-brickos-coff/include/brickos/v0/lnp/sys/*.h
./h8300-lego-brickos-coff/include/brickos/v0/lnp/*.h
./h8300-lego-brickos-coff/include/brickos/v0/rom/*.h
./h8300-lego-brickos-coff/include/brickos/v0/sys/*.h
./h8300-lego-brickos-coff/include/brickos/v0/*.h
./h8300-lego-brickos-coff/lib/legacy-0.9.0/lib*.a

As a point of comparison, an installed "bibo" multi-kernel instance with headers v1 would be structured as follows:
./h8300-lego-brickos-coff/boot/bibo/bibo.[lds,bin,coff,srec,lxMakeconf]
./h8300-lego-brickos-coff/include/brickos/v1/rom/*.h
./h8300-lego-brickos-coff/include/brickos/v1/sys/*.h
./h8300-lego-brickos-coff/include/brickos/v1/*.h
./h8300-lego-brickos-coff/lib/bibo/lib*.a


More Detailed Explanation:
The brickOS-bibo project has been intended as a v.Next (and forward) of the original brickOS project.  Apart from what perhaps might be considered a more real-time approach with respect to the time slices in brickOS, the capabilities that existed in 0.9.0 should still be available in brickOS-bibo.  In general, I would expect the feel to be familiar.  If there are specific backwards-compatibility gaps, we can certainly look to bridge those or provide an option in config.h.  (For example, I'm not sure why bibo dropped all the C++ headers and nearly all C++ demos, but offhand I'm not aware of any specific reason that might preclude their inclusion).

A little bit of background and history is at the links below, but the short is that the initial work in this brickOS-bibo project was started before bibo's initial release but was later revised to make what was bibo 0.01 the baseline and does incorporate the source code diffs between bibo 0.01 and 0.02:
* An initial set of 14 patches for brickOS 0.9.0
   - https://sourceforge.net/p/brickos/patches/21/
* A revised and updated set of the above patches for brickOS 0.9.0 by Carl Troein (16 total patches)
   - https://sourceforge.net/p/brickos/patches/22/
* A reworked and expanded set of 38 patches, bringing various brickOS patches targeting various brickOS versions to bibo 0.01 (patch 21 in this set includes the source code changes that updated bibo 0.01 to 0.02)
   - https://sourceforge.net/p/brickos/patches/23/

Breaking things down further—
COMMANDS / HOST UTILITIES:
Specifically regarding the commands, I would expect the host utils from brickOS-bibo to still work with brickOS 0.9.0.  While the latest host utils offer additional features (such as enabling use of TCP as a TTY option), offhand I am not aware of any breaking changes in those tools—firmware transmitting, *.lx program writing, program transmitting, LNP communication, etc. should (as far as I know) work across versions.  Two command names were modified, but updating the Makefiles that get installed by brickOS 0.9.0 to use these new command names should allow the same commands to be used by both.
* firmdl3 is now just firmdl
* makelx, which was not really a user-facing command, was renamed to writelx so that a "makelx" script (playing off the more common and familiar "make" command name) could be applied to building *.lx programs.

KERNEL AND DEMO PROGRAMS:
While some APIs in the include files changed a little as additional capabilities were introduced (like motor), some of the old APIs could potentially be mimicked with some #defines in the includes.  In a few other places, visibility of / access to internal data structures is now better controlled, so any use of such internals in user programs would need to be updated to use the "public" APIs.

If using the default kernel configuration, another notable change is that ASCII is now disabled by default, but it can be enabled by users who really want it.  In practically every case reviewed, ASCII seemed nonessential and could be easily replaced by using the much more space efficient CHAR_* to print to the LCD instead.  (I'm also looking at some additional backwards-compatible size reductions around the music notes.)

LIBS:
As long as the configuration has been setup accordingly, I expect the latest libs source code to be usable across versions.

ALL FILES THAT VARY DEPENDING ON THE SPECIFIC KERNEL CONFIGURATION:
Except for the include files, files installed for the default brickOS v0.9.0 configuration could perhaps be assigned the kernel name "0.9.0" and installed side-by-side following the new multi-kernel path conventions.  The Makefiles and makelx script installed by brickOS-bibo still generate *.lx files, so as long as the appropriate *.lds file is still installed alongside the *.srec file, they would likely work with brickOS 0.9.0, too.

INCLUDES:
These have been addressed by making the "0.9.0" includes a "v0', bumping the brickOS-bibo version to 1.*, and then versioning the current brickOS-bibo includes as "v1".  An additional *.lxMakefile file has been added alongside the installed firmware files that includes the major version / version group (currently either "v0" or "v1").  , after which Makefile.lxprog could be updated to also check that when looking up other kernel-specific information (like BASE1 and BASE2) and then set the include file path accordingly.  Kernel names have to be unique across all versions, otherwise this particular approach wouldn't work.


Regarding BrickEmu configuration challenges:
A more robust configuration for brickEmu would appear to duplicate in a number of aspects the configuration that already exists for brickOS-bibo.  For example, GUI.tcl hard codes in "-l*" form the kernel libraries used; it needs to know the path to those libraries; it looks for a ROM file in a specific place; cross-tools path/name aren't "discovered" by configure; it needs to know where ir-server is….  With the brickOS-bibo Makefiles now more modularized, it should still be possible to make the emulator aspects modular, too.


> I would be interrested for a new release with improvement on your main
> branch

Happy to oblige.  :-)


Thank you,
Matthew