Dissapointing

Well time goes on at a rapid pace, and people who tell you it goes faster as you get older aren't lying. Its been a particularly hard year this year, with the issues surrounding covid in the background, but AROS still moves on steadily as ever.

One thing that has left me disappointed though, is unfortunately stumbling across "AROS Exec" - a forum I used to think of as home, back when it had a friendly respectful community. Its a shame those days are long gone. The place is all but dead with only a hand full of die hard, vocal, "anti ABI v1" fanboys spreading their long tired stories about how bad ABI v1 is, remaining - and chasing any prospective new members away with their rude attitudes and bad information.

We once tried hard to keep the Amiga troll like characters out of the forums, because we knew it would end this way - its just sad to see it has.

Equally as disappointing is the disparaging remarks found there about the hard work being put into AROS by the developers. ABI v1 is both praised without acknowledgment whenever the improvements and fixes are backported to the "ABI v0(*)" code they are using, attributing the improvements and fixes to the people doing the backporting  - and also claimed to be an unstable mess with little done to improve it.

Despite the huge amount of debugging, number of fixes to the underlying system, improvements to cater for running on m68k and newer pc hardware (and fixes to allow AROS to boot on considerably more systems than it ever did in the past), They make statements about how ABI v1 hasn't had any work or improvement done - in part because they are already using the improvements that were not meant to be part of ABI v0 under the Frankenstein version they do use, and don't attribute the improvements to the AROS Developers but instead to the distro maintainers for backporting it.

It really is sad to see people so tied up in a false narrative that they cannot even see how foolish the path they go down is.

I'm glad this year is almost over,

Going back to AROS though, I can not wait to see what the next year brings. There has been some recent interest in updating the localization for a number of languages - something that has always been an annoyance for me.  Unlike code, you cant just jump into another language and correct things or google translate will make a complete farce of it - and despite the number of supposed users and developers AROS has had from all over the world - very few show the interest or dedication in making sure their language and localization is properly supported,

We made some changes this year to separate the actual translations out of the code base, because it was apparent some people where interested in contributing but didn't know where to begin in the bewildering source tree that is AROS. To cater for this, all the translations are now handling in a sister "site" - the AROS Translation Team, on GitHub.  This allows people who only want to help towards supporting their language, can easily contribute and have the improvements included with AROS over time. There is still a long way to go until using it is comfortable, and we have a nice set of guides for people to get involved - but its a start in the right direction.

LLVM is getting closer and closer to building AROS completely, with only a few smaller niggles still to iron out before the first testable binaries should come off the press, however hosting nightly builds for it is proving somewhat of a problem. Just building the LLVM toolchain takes more time than most complete AROS builds including the GNU toolchain, and need massively more space - more than the virtual environments hosting the builds provide in fact. To really address this needs a bit of work in AROS, but it is lower priority than other work currently underway. It has already been discussed over the last 2 years, about breaking the builds up so that the toolchain is not only built separately, but made available for download once built. We also discussed addressing the lack of proper SDK for AROS, which would be a necessary component with the toolchain, to produce binaries suitable for use. A little work has been started on these things, but the effort needed as mentioned, is lower priority than fixing the current issues.

One annoyance, that I have been investigating for a while, are the lockups during boot some people experience. They tend to be one of these very hard to replicate ones - only happening in certain hardware configurations, or driver configurations. So far Ive managed to fix a quite a large number, some of which that have been in the code base since 2000 or before! they include (but aren't limited to..)

# Code passing handles to stale data on the stack around, dating back to the early 2000s. Even intuition was doing it leading to some bizarre results.
# Hidds using the same attribute ID's for different classes attributes, due to the way they where allocated. While it doesn't sound obvious - this was very hard to debug. it only happened if the drivers started and some failed in specific orders, until eventually one driver would try accessing an object using different attributes that had the same value - confusing the life out of the code trying to figure out what it was meant to be doing. In the best case it had benign consequences, but at the other end of the scope it could end up trashing memory or just crashing AROS for apparently random reasons.
# ATA device just doing things because it wanted to try regardless of the consequences, even when the hardware was already shouting about things not working. This one is a common reason for AROS not booting, when it tries to talk to ghost devices and ends up hanging forever,
# the PS2 keyboard/mouse driver was conflicting with timer.device accessing the PIT timer, causing all sorts of weird erratic behaviour - or locking up for some people. ive rewritten the PS/2 driver now to use the timer.device itself, resolving the problem completely.

.. and then there are the ones I am still working way at.

A few years ago we introduced code to change the interrupt handling in AROS.  Up until that point, AROS had only supported the legacy PIC's interrupts. This was a bit of an issue, because we wanted to bring up other CPU cores now and also look at using other newer PC hardware features (MSI interrupts, etc). A lot of effort has been put in to rework this so that we now use all of the CPU's vectors, and have separate "drivers" for the different IC's in a PC. We have code for the old legacy PIC still, that can be used on its own or as a fall back, but we now also have an APIC interrupt controller that registers the remaining device IRQs and allows us to expose functions for routing, and modifying them outside the kernel. We also have an IOAPIC driver - but this one has been the cause of a few headaches to say the least.

According to the Docs, you set up your APIC, parse the ACPI tables to figure out what IRQs are in the system and how they should be delivered, find the IOAPIC(s) and configure them. Sounds Easy? Well after coding all that it appeared to work great .... except not everywhere.

It turns out even though its easy enough to set up the IOAPIC stuff using the information provided - a lot of systems lie about the PIT Timer, and how it is wired up. And this is where the problems occur. On some systems, as soon as the timer is used - it goes haywire resulting in an IRQ storm, so AROS doesn't do anything except continually handle the timer interrupt. Ive been scratching my head about this for the past little while, because it isn't very well documented (if at all) how to handle it. Sure, you can disable the IOAPIC (and aros has a "noioapic" boot option for this purpose), but how to make AROS work correctly with such flaky hardware?

One thing I tried, was changing the delivery details for the IRQ. I spent a considerable amount of time writing code to try and detect when the storm happened, and when it did - reprogram the route for the IRQ, and then let that run for a while and see if it fixed it. Suffice to say ... it didn't. I tried all 4 combinations of delivery options with the same result ... so what can it be? to be continued ....

Another problem that is more concerning is that AROS has a long standing and very hard to trigger issue where it loses tasks. Again, it seems to only really occur during boot - and only under very specific conditions which I have yet to pinpoint. When it does, one of 2 things happen.

#1 When input device is active, some interrupt occurs signalling a task. Input device resumes but it never switches away ... ever. I wrote a tool for HyperV that let me break into AROS when it hung, and dump the task lists, and sure enough there where tasks ready to run with pending signals, but for some reason input.device wasn't for letting go.

#2 some interrupt occurs from a hardware input device signalling "input.device" to run. The schedular switches out the running task ... but input.device is no where to be found ....

Again, Ive invested a massive amount of time and effort investigating this one so far but to no avail. From the schedulers point of view everything is working as it is meant to - but input.device has mysteriously been taken out of the picture.

Until I can get to the bottom of those 2 issues, I have very little interest on working on much else on AROS. I have an nvme driver in the works, which is basically complete - but debugging it is becoming tedious due to the random previously mentioned bugs.  the more debug that is enabled, the more likely AROS triggers one of them.

So that brings us to this next year. Atleast for the foresee-able future, I will be continuing to focus work on addressing the boot/stability issues already mentioned - and any others identified in due course. I'm hoping I can get them addressed relatively soon, because we really want to try and get the RasPI build working again (Ive recently fixed some build issues, so now there is just one problem linking the Roms remaining), and get the toolchain/SDK side addressed, and move forward with fixing the non-system applications (particularly posix like code) that seems to be unstable currently. All of the main AROS components themselves are running extremely stable - to the point that sometimes I find myself just launching them to reflect on how bad they behaved before - but until we have all our user land software running again I wont be happy.

... Lets hope this next year is a good one

(*) - It is hard to claim they are even running ABI v0. A decision was made by the AROS developers long ago before IcAROS, or others distributions where started - that due to the lack of any "real" ABI definition, and due to needing to make drastic ugly changes to the code to support binary compatibility with m68k, we would need to make changes to AROS so that we could clarify what the ABI was and not end up with an ugly mess of code to support the new targets.

Because this would likely lead to binary compatibility issues, and other breakages to cater for proper usage of the ABI, it was decided to snapshot the repository at that time and call it ABI v0 (as in it has no ABI), and move forward under ABI v1.

So regardless of what nonsense is spread on forums in a childish manner - ABI v1 "is" AROS.

Now, unfortunately, to appease the community that had grown around distributions built using ABI v0 - the whole reasoning for having ABI v0 a distinct separation was breached. Changes where backported breaking compatibility with the "existing" ABI v0 code anyhow, and not on one occasion - but almost as often as a new IcAROS release was put out. Why? So they could both claim it was ABI v0, while using all the changes that are meant to be part of the distinctly separate ABI v1 due to the breaking compatibility. You cant make this stuff up.

 

Ahh how time flies ...

Well its been a long 3 years since my last post, with a few absences due to hardware issues amongst other things. I'm still not in a position to work on the things I would like, but due to a few people pestering me when I had decided to stop working on AROS, i've actually still been working away behind the scene (mostly to address issues wawa was encountering, such as missing atomic support to compile certain c++ code on m68k and arm).

One of the main things I have been spending a lot of effort on during this time is reworking the AROS build system so that components get exactly the flags they should for a given toolchain set (e.g. host, kernel and target), because in the past it has been very hit and miss on which ones the components end up using resulting in subtle differences in how the code is compiled in different environments.

After a good 3 or 4 years of hacking away at this I now feel like there is starting to be a light at the end of the tunnel, and feel confident that the components are now using the flags they should be (bar some minor fixes still needed in some specific components mmakefile's).

Another part of this was splitting up the flags passed to the compilers/assemblers so that they are in the correct variables rather than being all crammed into the CFLAGS, making it difficult to work with. Going forward it should be easier to manipulate the used flags so that e.g only the ISA components get replaced to compile for a different flavor of processor on a given platform, and this is currently used to make sure we get all the correct flags needed for m68k builds targeting 68000/020 etc cpus or x86 32bit and 64bit code generation.

Correcting this has allowed the darwin builds to be re-enabled (although due to their lack of maintenance, they probably still need fixes in the code base to get working..)

In combination with these fixes and changes (there is also better c++ handling in the templates, amongst other things),  I've also updated the AROS toolchains through to the current experimental version 9.1.0 - which is being used to build the pc-x86_64-smp target. So far the toolchain has been successfully tested with all the AROS targets, bar PPC - although the ARM support is needing a bit of adjustment, which i'm currently talking with michal to get done.

Well, if you thought that might be enough to keep me busy - ive also been working on other things!

The m68k target has received some love to try and improve its performance and behavior a little, and address some problems using it on vampire - though this work is still ongoing. The uae generic gfx card code has been reworked into p96gfx.hidd and handles other devices better now, has fixes to the sprite imagery handling, and should work with multiple p96 .card driver instances at the same time. There is also preliminary code in place to detect remapped rom images so that AROS does not try to use the physical memory they occupy (e.g when booting an a3000 using superkickstart).

ATA device has had its api adjusted slightly so that it uses generic definitions for the controller/bus/unit interfaces inherited from the storage subsystem classes.  AHCI devices has also been adapted to expose the same api''s and classes so they now operate using the common storage subsystem, allowing them to be enumerated easily in sysexplorer etc.

Finally, this summary wouldn't be complete without mentioning that I have updated our MesaGL subsystem from the quite old v7 we have been using since its initial inception, to the current v19! Unfortunately (and after discussion with other devs) it meant that for the time being the nouveau driver had to be disabled since it needs considerable work to be able to compile with the newer gallium code base, but if someone decides to do the work before Michal or myself can get round to it - we will finally be able to use modern 3d graphics technologies in AROS!. Im really quite excited about this, and cant wait to see the first OpenGL 4.5 code running native on AROS - and maybe even Vulkan down the line...

Well, Its been a while....

As ever hacking on AROS has proving to take up much of my time, too much to even think about posting on my blog. But here goes ..

Since the last update ive been hacking away on a number of things, mainly my Gfx subsystem update. So far I have most drivers adapted but need to finish implementing the software cursor support before they can all be used properly.

The changes themselves comprise of a number of things -:

# Separating the gfx.hidd into a hardware driver and display parts. This is to allow one gfx device to expose multiple displays to AROS, and has involved a number of changes to how the gfx subsystem works internally. Hopefully sometime in the near future I can make a more detailed post about the changes.

# Making the gfx hardware drivers real "AROS" hardware drivers - that way they can be enumerated using the hardware subsystem and exposed in apps such as sysexplorer.

# Adjusting all the gfx components and stuff which touches the affected interfaces to work again.

# Adjust the intelgma driver to use the base software compositor - its actually faster and has more features than the one that is embedded in the driver (and is very out of synch with the main code)

There where also a few bugs encountered in the gfx susbsytem that after fixing make some of the planar display modes work better (especially on hi-color displays).

I'm happy with the changes so far, and hope to get the remaining stuff done and pushed to the AROS repo as soon as possible.

The next thing I've been playing with is updating mesa. So far I have the IntelGMA driver and software rasterizer built using the newest mesa code (as well as the core mesa components), but until the nouveau code is updated cant push the changes to SVN.  Testing with IntelGMA and the Gfx subsystem updates yields roughly a 20% improvement in performance on Intel atom based systems which isn't too shabby.

Ive also done some massive work to the build system over the past 2 years so that we can finally now build working binaries with the latest GNU toolchains. Another part of these changes is to prepare the build system to use other toolchains (LLVM), however there is still some work on the toolchains themselves before this can be taken advantage of.

My final project though, has been an interesting ride. I've been working on getting the FinalWriter 6 codebase to build with gcc, aswell as fixing it to run on AROS/64bit.  It was surprisingly easy to get the actual code compiled but took a few weeks to dig through some of the issues caused by using a different toolchain than it was originally written for, and finding the issues stopping it from running on 64bit. Evenetualy we managed to bring it up though, and while it ran there where still some problems and visible gfx issues.

Over the past week or 2 I have been working on adapting the GUI code to use localization via the locale.library, and have been quite productive with it (about 80% done, including the german translation!). So at the weekend I decided to tackle a few of those Gfx niggles. It was an interesting experience, and after some soul searching and head scratching I managed to get to the bottom of them!

A bit of code fixing later and the rendering is now correct. There was a bug needing fixed in AROS's vsprite/bob handling, which resulted in the pop up menus displaying incorrectly, and also  the base finalwriter shell class used hard coded GUI elements which required a bit of reworking (as in adapted to be a subclass of sysiclass) but finally the results spoke for themselves! it now has AROS's checkbox, radio and (themed) window scroller arrow gadgets, making it fit in much better with the rest of the system.

Well, It has been a few years yet again since I last posted on here. So I felt it might be time for an update.

Over the past year I've been having some fun working on porting FinalWriter to AROS (with massive help from Jason) - and it seems to be pretty much done with some minor issues still to resolve.

I've also been working on A number of other AROS projects including -:

# Updated my Doom3 port to the newest code, with some fixes that seem to finally resolve the known crash, as well as allowing the use of the expansion - Resurrection of Evil!
# A little more work on the RasPi port with Michals help, including starting work on supporting multiple cores within the scheduler.
# Rework the AROS build system to really use its own cross compilers with some related tidying
# fixes to the AROS headers so that we can use them with newer C++ standard compilers
# update the Gnu toolchains so that we can now build gcc 5.1. Also cleaned up some clang related issues.
# Adapt deadwoods hostgl library to run on ABI v1, and reworked the gl.library base implementation to be a proxy to the "desired" gl implementation (set using an ENV variable). This means there should be at least 2 library's on a system to support gl - the gl.library stub and the real implementation (hostgl.library and/or mesa3dgl.library).
# Change how AROS's gfx hidd subsystem handles the creation/destruction of some base "object" classes, so they use a unified (and driver overridable) way to implement them. This means drivers can also expose new object classes not exposed by the gfx subsystem itself, eg gallium contexts.
# Updated (privately) the Mesa GL implementation to v11, including minor changes to how the gallium interface is handled. mesa/gallium no longer use a hard coded list of possible classes to try, but instead ask the screens gfx driver to create a gallium object, and if this fails fallback to the software implementation.
# Update the softrast, and intelgma drivers to use the latest gallium driver code. This finally brings the software rasterizer to a level where it correctly displays the doom3 logo on its intro page.
# Adapt the AROS gfx hidd subsystem so that the gfx drivers are exposed as real hardware drivers - e.g. so that they can be enumerated in the sysinfo tool.
# Adapt the AROS gfx hidd subsystem so that the gfx "driver" objects which represent an individual hardware instance (e.g on die GPU, or discrete controller) expose one or more display objects each with its own mode array database etc.
# start adapting the way AROS exposes its software cursor so that it no longer uses the fake gfx subsystem, but instead the base display class exposes the behaviour.
# adapt the compositing code to use the changes, as well as make the intelgma driver use the base software compositor.

There is still a LONG way to go on the Mesa side (updating the remaining drivers) as well as fixing the remaining issues with the gfx subsystem changes, however the parts working/adapted so far seem to do what it says on the tin. the glsimplerendering test on my Atom's Intel GMA chip has gone from 700to800 FPS up to 900+.

I'm also starting to try and gear up to implement the next parts needed to get my "new" wanderer code base usable. This primarily means re-implementing the icon handling.

The first part I really need to tackle is the base/public "Iconlist" class from legacy wanderer. This is going to be replaced by a few Public classes that are meant to be very basic and usable outside of wanderer also. They don't explicitly represent a list of files, just any old icons which have a label. The most basic units are the IconClass, IconlableClass and IconimageClass. As it says on the tin, they represent the Image for an Icon, the Label for an Icon, and the Icon that exposes those (N.B: an Icon can have more than one "label" entry) The next class is the IconFamilyClass. It exposes a list of IconClass objects, and handles sorting them (the base class is only aware of the icon's label and co-ords). After this come the IconListClass itself. It exposes the input event handling for the IconFamily list of icons, and related codes.

Since these are meant to be very generic - the classes do not handle any IO or related things - they are exclusively interested in a list of "icons" and nothing more. This means they could (should) be used to implement e.g. Toolbar classes.

At some point wanderers own Toolbar panel will get the iconlist to handle most of its functionality, allowing for sorted/positioned icons and fine tuning the appearance. Once this is done it will be time to replace the "private" wanderer versions of these classes so that they tie up the configuration/appearance to suit wanderer, and so that they can populate the iconlists using the new "location" classes exposed by wanderer, matching the entries to icon objects. This will finally enable the new wanderer branch to display the desktop and local filesystem contexts again, but now using the new API's.

... lets see .... where to begin ...

Ahaa

Thanks to deadwood (Krzysztof Smiechowicz) ive managed to get AROS booting off my Atom330 board. Seems the BIOS requires an active partition on the drive before it will boot it.. strange but, oh well, it works =)

Been a while

Yes, as ever ive been neglecting my blog.

Having a 1 year old son is very time consuming im slowly finding out

Anyhows I recently got hold of an Intel Atom 330 based board for AROS thanks to ClusterUK with the intention of writing a NIC driver for it .. which im slowly working my way through (RTL8168/8111). There have been a few setbacks - primarily AROS wouldnt boot on the box.

After a lot of searching and some insight from Error (Tomek Wiszkowski) it became evident that the ata.device was finding the ICH7 AHCI S-ATA controller, but couldnt use it. Not only this, but since it had found the controller it decided not to add legacy ports, which resulted in AROS finding no bootable devices. Luckily it was trivial to fix the issue and now AROS is booting happily on not only the Atom 330 board, but a whole bunch of other users ICH systems which had failed after ata.device got "upgraded".

Another issue was present which resulted in the early startup menu showing everytime the machine was booted, and couldnt be exited. Thankfully this was also relatively easy to identify and fix so now I can boot into wanderer quite comfortably - and have Installed AROS on a 200gig S-ATA drive attached to the system.

Theres still a peculiartiy though that makes it unable to boot the system (not sure yet if grub isnt installing to the MBR correctly, or some other issue is preventing it loading) -so I will have to look into this when I find time ..

.. but for now its onwards with the rtl8168.device ..

A new home ...

Ive got round to setting up an SVN repository on google code, where I will work on projects for AROS that I dont want to do in the main tree..

Im in the process of cleaning up the mesa6.5 dirs to import it, so that should be available with smbfs etc shortly.

see http://code.google.com/p/kalamateesarosextras/