NodeOS

Now that almost all the tasks for the [alpha](https://github.com/NodeOS/NodeOS/issues/37) are complete, I think it's time to start thinking and discussing where to work now: - [ ] UIs - [ ] blessed correctly working by default - [ ] NodeOS font - [ ] HTML renderer (Chromium Embedded Framework) - [ ] OS flavours - [ ] mainframe - [ ] workstation - [ ] alternative kernels - [ ] FreeBSD - [ ] runtime.js - [ ] alternative platforms - [ ] ARM (raspberry pi) - [ ] MIPS - [ ] splitted modules for each layer - [ ] tests for all packages Any other idea? :-)

We... sort of :-P [](http://xkcd.com/1508/) It could be [runtime.js](http://runtimejs.org/), too... ;-)

It's has been a tough week, but as we advanced on [our previous post](https://github.com/NodeOS/NodeOS/issues/118), finally we got a great milestone leading to [our first alpha release](https://github.com/NodeOS/NodeOS/issues/37) to raise lower the entry barrier to new NodeOS users: now NodeOS has [pre-build images](https://github.com/NodeOS/NodeOS/releases) automatically generated both for QEmu, CD live ISOs images ([one of the most requested issues](https://github.com/NodeOS/NodeOS/issues/30), also on [Twitter](https://twitter.com/espectalll123/status/566359430472957952)), and disk images (with a writable usersfs partition! permanent storage! yay!:-D )...  ...that work on real hardware!!! :-D To create the CD-live ISOs it was more-or-less easy thanks to the fact that now the root filesystem is read-only (so it can boot from a CD-ROM) and using [grub-mkrescue](https://www.gnu.org/software/grub/manual/html_node/Invoking-grub_002dmkrescue.html). Problem is, that it requires to define a usersfs partition by hand, or if not the changes would only be done on memory. To allow to make them persistent the best option was to make a partitioned disk image with both a read-only rootfs and a read-write usersfs, and allow to write it to an USB pendrive. At first I though about continue using grub, but after two days I left it: it's difficult to use on disk images, bloated and require to use ```sudo```, so I decided to [think different](http://en.wikipedia.org/wiki/Think_different) (I did already by using [musl](http://www.musl-libc.org/) instead of [glibc](http://www.gnu.org/software/libc/)) and ended using [syslinux](http://www.syslinux.org/wiki/index.php/The_Syslinux_Project). Result? Easy to configure (and similar to the one used on the Raspberry Pi boot process), works natively with disk images, don't need ```sudo``` and it was ready in just two hours :-D I'll later from grub-mkrescue to use [isolinux](http://www.syslinux.org/wiki/index.php/ISOLINUX) so I can use only one boot system. Oh, and if you want to write it to an USB pendrive, t[here's also a tool to do this](https://github.com/NodeOS/NodeOS/blob/master/bin/installUSB) and automatically expand the full disk space... Would be interesting to see this on a hard disk... ;-) There are some more issues to fix. For example, the errors you see on the previous image are regarding that the image is prepared to mount usersfs from ```/dev/sda2```, while as you can see it has set the USB drive as ```/dev/sdb``` (and ```/dev/sda``` is my internal hard disk), so I'll need to add support on [nodeos-mount-filesystems](https://github.com/piranna/nodeos-mount-filesystems) to mount them based on UUIDs to prevent this. Another more annoying was that the first test on real hardware on [my work](https://www.kurento.org/) MacBook didn't booted at all, just to get to the point later that Macs are using EFI instead of BIOS, so I'll need to investigate how to do a disk image that can boot on both systems. Anyway, now I can say it:  :-P

One of the advantages of being based on NPM is the so modular that NodeOS is. That's the reason why [you would have not seen too much movement lately](https://github.com/NodeOS/NodeOS/commits/master), but the fact is that in the project dependencies [there has been a huge improvement](https://github.com/piranna/nodeos-mount-filesystems): now thanks to a huge and agressive usage of the new Linux kernel multi-layer OverlayFS (we are needing to use a release candidate version of upcoming Linux 4.0 so [it could be available](http://www.phoronix.com/scan.php?page=news_item&px=Linux-3.20-OverlayFS-Changes)! :-D), NodeOS offer a per-user independent & isolated root filesystem, and in fact now there's no real root partition at all (that's great to simplify the process to create a live ISO or a Raspberry PI disk image!!! :-D), making this not only the first OS to have a similar approach (loosely inspired in [Plan9 ```bind``` union directories](http://en.wikipedia.org/wiki/Plan_9_from_Bell_Labs#Union_directories_and_namespaces)) but probably making it one of the most secure OSes, at least from a file access point of view :-) ```Javascript [ 1.072752] EXT4-fs (sda): warning: mounting unchecked fs, running e2fsck is recommended [ 1.076131] EXT4-fs (sda): mounted filesystem without journal. Opts: errors=remount-ro Hello! I'm a user init script :-) Welcome to NodeOS!: username: nodeos Welcome to NodeOS!: password: / # ls [ 'bin', 'dev', 'etc', 'init', 'lib', 'proc', 'share', 'tmp', 'usr' ] / # ls /bin [ 'blessed', 'cat', 'cp', 'ip', 'ln', 'ls', 'man', 'mkdir', 'mv', 'node', 'npm', 'nsh', 'pwd', 'rm', 'sh', 'slap' ] / # node > process.env { HOME: '/', TERM: 'linux', LANG: 'es_ES.UTF-8', vga: '0x318', PATH: '/bin' } / # cat init #!/usr/bin/env node console.log("Hello! I'm a user init script :-)") / # ./init Hello! I'm a user init script :-) / # ls usr/bin/ [ 'env' ] / # cd usr/bin/ /usr/bin # ls [ 'env' ] /usr/bin # cd / # ls share/ [ 'man' ] / # ls share/man/ [ 'man1', 'man3', 'man5', 'man7' ] / # ``` Pretty, isn't it? :-D The way it works is fairly simple and it's inspired by [this IBM article about mount namespaces](http://www.ibm.com/developerworks/linux/library/l-mount-namespaces/index.html) (that definitely are the next step where to move to improve all this OverlayFS big hack... if needed :-P): ```initramfs``` not only mount the users filesystem, but also means as a basis for the "general root filesystem" used as basis later for all the users (since by design since Linux 2.6.0 it will be always loaded in memory, so we re-use it). This is done by removing the unneeded modules and executables used to start and mount the filesystem so earn some RAM, but also to remove some possible security holes on that modules, so we keep only with an initram only with the Node.js executable and its needed dynamic libraries, and since they will be used everywhere this makes also the system faster. The users partition need to be mounted somewhere, so I'm doing it on the ```/tmp``` folder on the initramfs, so this way I can later mount over it an in-ram ```tmpfs``` filesystem for each user, hidding the raw users partition from being accesible by anybody and also preventing to write temporal data on the real hard disks. After that, we use this initram as a read-only "carbon copy" where to mount over each of the users home (also root) and later using [chroot](https://github.com/melor/node-posix#posixchrootpath) to give to each users an unique root filesystem isolated from the others. This is only needed to exec the per-user ```/init``` executable and when login in the system, so it's easy to have the access controlled, and all their subsequent process will be running inside their corresponding user ```chroot``` jail. There's a minor exception regarding to the ```root``` user: if it's available in the users filesystem (that's not mandatory at all), since [logon](https://github.com/piranna/logon) needs to access to the users home directory to read their login configuration, it is mounted first and later the users partition mount point is moved to a ```home``` folder inside its own filesystem, so it can access to them without modifying the carbon-copy initram root filesystem. After that, it is configured exactly the same way as any other regular user. This also has the advantage to give to the users a simple filesystem hierarchy were to work, since their "home" is in fact the root of their own filesystem hierarchy, so they can easily install packages globally without requiring special permissions and will not need to configure anything since they will naturally be installed in their home directory by deafult. There are some improvements that need to be done, obviously. First of all, the "demo" users filesystem that's automatically generated on the build process needs to give diferent ```uid```s and ```gid```s to each one of the users folders since this is used as basis for the security scheme (maybe a patch for [genext2fs](http://genext2fs.sourceforge.net/)?), but since it's only for demo purposses this is not so a problem (but definitely we need to add some some info about how to create and config a custom, real one...). Other notable problem is regarding to the ```/dev``` and ```/proc``` filesystems, since they are shown on each user filesystem as empty folder. In the case of ```/dev```, the best solution is [to implement a filesystem that only show the device entries that are accesible by the user and owned by him or not currently used](https://github.com/NodeOS/NodeOS/issues/95), so they could be exclusively accessed. This could be easily done by using [fusejs](http://c4milo.github.io/fusejs/), since the NodeOS Linux kernel is already being compiled with support both for [FUSE](http://fuse.sourceforge.net/) and [CUSE](http://lwn.net/Articles/308445/). Regarding to ```/proc```, something similar could be done, but seems its configurable at mount how accesible it is, so by using shared mountpoints I could have it replicated on all the users root filesystem. On the other hand, I would like that [Node.js remove its dependency on it](https://github.com/joyent/node/issues/10426) so it could be removed at all from NodeOS, since it's obviously a posible entry point for security issues since you hace access to all the kernel internal structures... :-/ Next step, update the project documentation, some love for Docker (I have been focusing on QEmu since by default Docker needs ```sudo```, so it makes more dificult to build the images) and generate (nightly) live ISOs :-) It will be funny to see NodeOS running on real hardware... :-D P.D.: oh, and [we have now 2200 stargazers](https://github.com/NodeOS/NodeOS/stargazers) and counting!!! :-D

In the last days there has been few work here at NodeOS, but that's because [we has been worked on a side open source project](https://github.com/NodeOS/NodeOS/issues/105): thanks to the awesome work of @formula1, now GitBlog is a reality :-D [GitBlog](https://github.com/NodeOS/GitBlog) is a blogging system that use GitHub issues as engine, so it integrate directly with our current work flow based on them :-) We'll store the content of the legacy blog for historical purposses and reference. In the next days maybe there would be some updates, but at this moment is production ready, and in fact this will be the first post there :-D But GitBlog hasn't been the only thing we have been working on... the most important one has been a re-design of the overlay layers inspired by [this IBM article](http://www.ibm.com/developerworks/linux/library/l-mount-namespaces/index.html) that have bring us trhee important things: * root home folder moved to the users filesystem * users has their own local filesystem hierarchy... * ...and root filesystem now it's read-only :-D This has the great advantage that will be easier to generate ISO disk images and to port to other platforms, since the root filesystem is minimal. So minimal, that now it only host the ```resolv.conf``` and terminfo files, so in a future iteration we'll remove them at all. No root filesystem, only the initramfs to boot the system and everything stored in the users home! :-D A too drastical design? Yes, but we are not willing to create a POSIX system here, but instead only the required parts to create an environment were Node.js apps could feel like at home :-) This give us more flexibility to optimize CPU & memory resources (specially disk space) and increase security by having less components in the system. It needs still some work to clean-up the final users filesystem and about automatically assign a filesystem namespace to new process (at this moment they are all mount on boot...), but seems promising :-) In other things, some people think that a graphical interface is important for an OS. I don't think so, and in fact it complicate the overall system so we decided to [split NodeOS and have several flavours](https://github.com/NodeOS/NodeOS/issues/106), but it's true that would be cool to be able to use NodeOS to play some games... :-P and that's why [there has been some progress towards FbDev support](https://github.com/NodeOS/NodeOS/issues/39#issuecomment-76904769). We have it by default thanks to Linux kernel default configuration, so why not use it? We were thinking about writting a wrapper library and use it as basis to implement the [Canvas API](https://developer.mozilla.org/en-US/docs/Web/API/Canvas_API) on top of it, but the fact is that [node-canvas](https://github.com/Automattic/node-canvas) (the most popular Canvas API library for Node.js) has surpassed us and now [it start to support FbDev natively](https://github.com/Automattic/node-canvas/issues/533) thanks to our comments :-D It's cool when your work can help to move forward other projects... :-) And last but no least... now we have a [Facebook page](https://www.facebook.com/NodeOperatingSystem) :-P It's not active yet but we'll try to fill it soon :-)