I'm trying to locate ports of some of the more popular Unix/Linux utilities to other operating systems such as AIX and Windows. I find it helpful to have the same utility on all the systems I work on, not just on a few them.

Are you searching for other programs of this nature or creating other cross-platform utilities of your own? If so, please share your progress. It would be nice to add more options to this list.

cal
http://unicorn.us.com/cal.html

lsof for AIX
https://github.com/aixoss/lsof

top for Unix and AIX systems
https://sourceforge.net/projects/unixtop/

ntop for Windows
https://github.com/gsass1/NTop

lsblk like utility for Windows
https://github.com/tenox7/lsblk

simple cross-platform ping
https://github.com/sryze/ping

ps style tools for Windows
https://github.com/joeattardi/winpstools
https://github.com/katakk/pkill/

uptime
https://github.com/qwercik/uptime

experimental dd implementation for Windows
https://github.com/sryze/wdd

dd for Windows
http://www.chrysocome.net/dd

busybox-w32 has ports of several utilities that will work on Windows including dd, df, grep, ps, su and others:
https://github.com/rmyorston/busybox-w32

nano for Windows
https://github.com/lhmouse/nano-win
I also have a port of nano for Windows. It works with PDCurses.

An older version of htop was patched for AIX support and I've added a similar patch to a later version. When the htop project was contacted regarding patches they responded they were not interested in adding AIX support to the official version.

I have simple cross-platform implementations of uptime, nproc and free which I've been working on.
I've been interested in the BSD versions of diff and patch for a long while now. I often use a version of patch modified from various BSD patch implementations that were based on patch version 2.0.12u8. My variation includes some support for carriage return/line feed differences among systems. I've found it useful when working with patches from Windows or DOS systems that might accidentally introduce carriage return/line feed sequences instead of just line feed which POSIX systems use.

Recently saw an informative page covering some of the history of patch and diff:
https://invisible-island.net/diffstat/
It mentions that the 12u variant of patch contains no copylefted code. It also mentions a version 2.0.12.u9. So, I thought it was interesting that the various BSD versions found on NetBSD, OpenBSD and FreeBSD seem to start with version 2.0.12u8 as their basis. I'm very curious as to why they started with u8 instead of u9 but have seen no documentation on it. There doesn't seem to be that many differences between u8 and u9 so they probably didn't use much that was useful. If anyone knows anything further on this or know of some projects that started with the u9 version, I'd appreciate hearing about it.

BSD operating systems such as NetBSD switched from the BSD 4 clause license to the 2 clause. The 2 clause license is compatible with GNU license software while the 4 clause is not. So, it's not surprising that most of the code for BSD diff can be found with a 2 clause license. However, the diffreg.c file still contains the 4 clause license and a 3 clause license. I've searched for alternative versions of diffreg.c with other licenses. There are other versions available such as the diffreg.c that comes with the Plan 9 diff. They use the same algorithm. However, they don't contain some of the updates found in the BSD versions such as support for unified diff.

The BSD version of diff uses the longest common subsequence algorithm which performs in O(n2) at worst case and typically performs as O(nlogn). There is a newer algorithm by Myers that performs in O(nD). It's used by GNU diff. I saw a request a long while ago for someone to update the BSD diff to use the Myers algorithm. I don't think anyone's ever completed the task.

While the BSD version of diff is very usable, it would be nice to have a version of diff without the 4 code clause so that it could possibly be used as library code in conjunction with software that might contain GNU licensed source code. I prefer working with a more lenient license like BSD 2 clause or MIT rather than GNU when possible. So, it would be great to find a version of diff that supported unified diff output and avoided the GNU or BSD 4 clause licenses. I think the main alternative that might offer diff with a more lenient license and unified diff support would be Toybox. The Toybox license is BSD 0. However, Toybox doesn't support as many command line options as BSD or GNU diff.

I also just found out that there are versions of diff and patch for sbase. However, they're not in the main source code repository for sbase. You can find the source by searching the dev list from suckless.org. The sbase diff appears to be fully POSIX compliant. It's missing several features that were added to BSD and GNU diff programs. It supports unified diff only. The output is very close to the output of the BSD diff utility and it also uses the longest common subsequence algorithm.

I did run across the code to convert between diff standard output and unified diff output:
https://github.com/AceHusky12/unidiff
It's listed as public domain.

If anyone else is using an unusual variant of diff or patch or is writing their own, I'd be very interested in hearing about. I would love to compare notes on some of the different BSD variants out there and the various features they support.
I've been investigating some of the lightweight, command line utilities that are often used to check the status on a system. In some cases, it was hard to track down the packages they were in. It's difficult to search when some of the names are so ubiquitous. Just try running a search for the free utility.

I knew that many of the utilities were in the GNU coreutils package. I found out that many others were often supplied by the procps or the newer procps-ng packages. The procps and procps-ng packages are fairly Linux specific. So, it's not likely you'll find the code ported to other operating systems. While GNU coreutils ports to many platforms, I typically prefer GNU alternatives. They tend to be more lightweight with not as many features. Also, many GNU developers don't particularly like programming for portability and often won't accept patches for their code when there are portability issues. Many of the utilities I considered have been on Unix systems for a long time and are also available on BSD systems. The GNU versions tend to add more features and functionality. BSD systems don't use coreutils, so most of the programs are part of the operating system code.

Busybox and Toybox provide a lot of the utilities as well. Both Busybox and Toybox are designed as one monolithic program that can be accessed through links to the various utility names. That's convenient because there can be a lot of repetition between what some of these programs do. Having the code in all in one place means less maintenance for repetitive tasks. In a few cases, more standard versions of some utilities took this approach as well. For instance, the w program on a BSD system is also the who program.

It seems almost random as to what stats some of these programs cover. For instance, uptime not only gives you how long the machine was up, it also gives you how many users are on the system and system load averages. Sometimes seemingly non-related information may all be returned by the same function on a particular operating system, but sometimes it requires different functions to check each statistic. So, why put it together in this particular presentation? Likely, it's just because that's how it's historically been done and changing too much could break a lot of scripts out there. Some lightweight implementations don't always show all the information of the more common versions of the utilities either. For instance, some versions of uptime just show the boot time and don't bother with the other statistics.

Command line options for invoking a program may not be the same for different versions of a program. Also, since many of the programs have been around a long time, the command line options may not be that intuitive. For instance, I typically see -h reserved for help, but in some cases, it's being used for printing a human readable format. Some other utilities use -H for human readable. Some use both lowercase and upper case for human readable but use factors of 1024 bytes for one instance and factors of 1000 for the other. Personally, I would prefer to use -h for help and -B with a blocksize to indicate what factor I might want to divide by. That way you don't need separate command line options for kilobyte, megabyte, gigabyte and kibibyte, mebibyte and gibibtye. Many programs don't even offer options for terabyte which has become much more standard. As memory sizes go up, the older hard-coded options won't adapt as well. As mentioned, one of the issues with changing command line options to make utilities more uniform is that it could interfere with backward compatibility and cause other programs depending on that functionality to fail. However, all the various implementations aren't exactly standardized to begin with. The Open Group has some standards for utility conventions. Many programs use getopt or getopt_long to help standardize parsing of command line. However, it's not enough of a standard to prevent all the differences between different implementations. So, it's not like scripts can rely on all the same options to be there on different platforms or with non-standard software choices anyway.

I was interested in tracking statistics on memory usage, disk space, CPU usage, process and user related information and when a machine was rebooted. The tools I investigated included free and vmstat which show memory usage, df and du for disk space usage, uptime for CPU usage and boot time, nproc shows number of CPUs, ps and top for process information and utilities such as w, who and whoami for user related information.

I wanted to find lightweight alternatives that I could use on multiple platforms and not just Linux. Minix 3 uses a lot of the BSD utilities, so going with those might be one alternative. However, many still have a lot of code to implement them and the code is often very specific to BSD style operating systems.

One place to find lightweight versions of some of these utilities is ubase which was designed with the suckless.org philosophy in mind:
https://git.suckless.org/ubase/files.html
The code is very readable, but can be very Linux specific. Another alternative is nbase:
https://github.com/cheusov/nbase
This is a port of NetBSD tools to POSIX systems so that makes it a more portable option. Earlier versions of Minix had some lightweight versions of various utilities in their commands/simple directory before they switched to the BSD versions. PicoBSD also had some interesting lightweight utilities programs including aps and sps which are ps alternatives and vm which is a vmstat alternative.

There are also Windows ports of utilities such as ntop:
https://github.com/gsass1/NTop However, while some of these programs look like common Unix/Linux/BSD utilities, many are completely rewritten from scratch and would not port well to other operating systems besides Windows. Alternative operating systems such as hobby OSs that some developers create may have some of these utilities as well. After all, they're simple, command line based and useful. However, since many of the hobby OSs don't support POSIX or have large differences in kernel design, their implementations probably don't port well to other platforms either.

I'm currently in need of utilities that work on Linux, AIX and Windows. I've been investigating writing some of these utilities from scratch in a more portable manner so I can use them in a consistent way on whatever operating system I may need them. Unfortunately, they can be difficult to write, since the underlying functions are not part of the C or POSIX standards and are very platform specific. They can even change with the version of the operating system. Nevertheless, I'm going to continue to work on implementing more portable alternatives for some of the more common utilities. They're useful and their functionality is sorely missed on operating systems where they are not native. It would be really interesting to discuss design issues and trade-offs further with any developers who may be working on similar projects or with users testing out these types of utilities.

Would love to compare notes on this subject. Are there other simple command line utilities that I haven't mentioned that you use to check the status of your operating system? Do you know of other alternative implementations for some of the more common utilities mentioned? Are you interested in using some of these utilities on an operating system where they may not be as readily available? Let me know about your experiences in this area.
Was at a PHP meetup where the group was discussing Docker. Members and the presenter knew certain Linux distributions had a smaller footprint for use with Docker. I was surprised to find out they really didn't know why that was. One of the key factors is the C runtime library. Basic C runtime libraries just cover the functions and data structures that are part of the ISO C standard. Many C runtime libraries also add functions and data structures that are part of the POSIX standard as documented by the Open Group. Some C runtime libraries are rather bloated and provide a wide variety of functions (even beyond those documented by the ISO C and POSIX standards). Others provide a bare minimum. Some, especially those targeting embedded systems are designed for efficiency. Others are designed for functionality. Some provide no Unicode support (locale 'C' only). Some like musl, concentrate on UTF-8 support. Some try to support a large variety of characters sets and internationalization features. All these factors can affect code size and efficiency when compiling programs.

Alpine Linux previously used the uclibc runtime library and now uses the musl library. Most major Linux distributions use glibc. It was a big step, but a positive one when Debian (and Ubuntu) made the switch to eglibc. The choice of C runtime library can make a huge difference for the operating system.

For Windows developers, if you're using MinGW, the GNU compiler on Windows, you're using the Microsoft runtime libraries. The original version of MinGW used crtdll.dll but later versions use msvcrt.dll. Windows systems typically have one of these runtime libraries already installed. So while, you can't distribute Microsoft's libraries, you really don't have to. At least one is already on any particular Windows system. It gets even more complicated because different versions of Windows have different versions of msvcrt (such as msvcr70.dll, msvcr80.dll). MinGW uses a subset of the runtime functions based on Visual Studio 6.0. There are ways to access runtime functions from later versions of Visual Studio, but the application becomes less portable to various versions of Windows. Cygwin which also works on Windows, avoids the problem of dealing with multiple Windows runtime libraries and dlls and provides better POSIX compatibility for their runtime library by using a runtime library based on newlib.

There are a wide variety of runtime libraries designed for embedded systems. They're typical more compact and less bloated than a library like glibc and many are easier to port to various platforms. Of the runtime libraries available for embedded systems that are highly portable, I thought the newlib design was interesting. newlib is currently maintained by Red Hat. There are a limited number of syscalls (typically functionality provided by the kernel) that one needs to provide code for to get the library to work. Newlib uses a combination of public domain and BSD style licenses. Cygwin uses a runtime library derived from newlib, but adds several POSIX functions. It also uses a GNU GPL license. That means whenever you distribute programs linked with their runtime library you must also distribute the source code in order to be compliant with the GPL license.

As mentioned, the standard C library on most Linux distributions is glibc which was developed by the Free Software Foundation. The glibc project was slow to take patches and is a rather large library, so some distributions switched to eglibc which is binary compatible with glibc. The binary compatibility makes it easy to switch between the two. Some Linux distributions wanted to avoid bloated C libraries and work well on low resource or older systems. They chose uclibc which was designed for embedded systems. When I worked with a uclibc based distribution, I found myself making several patches to Open Source programs just to get them to compile. Many Linux distributions such as Alpine Linux are now using musl. It's designed for efficiency and standards compliance including full POSIX standards compliance.

musl was designed to work with a Linux kernel. So unlike choices such as newlib, it is not easy to port to other operating systems. The midipix project is endeavoring to create a POSIX compatible layer for Windows, so that musl will work out of the box on that system. Musl uses a MIT license. The midipix project will use a more restrictive license similar to Cygwin. Many basic embedded system libraries tend to use less restrictive licenses like MIT and BSD so that they will be adopted by companies. However, project or company adapts a C library to a particular system and adds a lot of functionality, they typically tend to use GNU GPL or other more restrictive licenses. Many projects dual license in hopes of selling companies a commercial license with less restrictions.

Google developed the Bionic C library for it's Android operating systems. It has a BSD license. It's also designed to work on low resource systems so it tends to offer less functionality than other C libraries such as glibc. It has partial POSIX support.

I've been hitting many limitations with the MinGW runtime libraries when it comes to porting. Alternatives such as Cygwin's or midipix's runtime libraries would overcome the issues. However, the licenses are much more restrictive. I know I definitely wanted more POSIX compatibility on Windows than MinGW offers out of the box. As a cross-platform programmer, it would be nice to take whatever C runtime library I end up with on Windows and reuse it on Linux and other systems. I looked at C runtime libraries for embedded systems which typically port well. Of those, newlib seemed the most interesting, because according to some of the documentation it only requires 17 syscalls to port it to an operating system. Some Windows CE compiler ports use newlib with added functionality for the Windows CE operating system to derive a working C/C++ compiler. When I investigated the newlib code, I did not particularly like the design, especially the way it handled threading by providing standard and threaded versions of functions. The implementation of file I/O looked like it had been modified several times and at this point could use a major refactoring. When I read some of the comments in that section of the code, I felt very uncomfortable using the library. I wanted a simple, clean, basic design that I can add to.

Another option I looked at was PDClib. I love the idea of a public domain library. MinGW original licensed their WIN32 and runtime code (which integrates with Microsoft's code) as public domain.
PDCLib was based on an earlier Public Domain library project originally at Sourceforge. I tried PDClib on Windows (which the developer says he uses it with), but I was unable to get file I/O to work properly. I contacted the developer to see if he needed with the project, but he really didn't seem to need any assistance at the time. PDClib only supports standard C functions. It does not provide any POSIX functionality. So, it would have limited usage as is for running most Open Source programs.

A number of original operating systems use their own C runtime libraries. I figure, if they can reinvent the wheel and create a C runtime library for their particular purposes, so can I.

I've been coding functions that are typically part of the C runtime library in order to provide better porting support for Windows. I wrote a C11 compatible thread library, several BSD and POSIX string functions, some POSIX file functions, etc. That left me with the dilemma of how best to integrate the additions with the runtime library. They really should be part of the library and part of the C standard headers. I currently have them implemented as supplemental libraries that have to be added separately. It's easier to test and integrate on various operating systems that way. Ideally, it would be nice to have everything accessible as one library though.

I'm very familiar in the various methods of connecting to the kernel in Windows. Some projects such as midipix just use ntdll.dll. Other projects connect to other Microsoft dlls such as kernel32.dll. One can use LoadLibrary, GetProcAddress to connect t a dll or if the library is already linked in, one can skip LoadLibrary. A few Open Source projects I've seen actually implement the LoadLibrary functionality from scratch. I'm not as familiar with the techniques to connect to the Linux, BSD and other kernels and would love to find more clear documentation on this subject. If you run across any good materials, please let me know ( http://www.distasis.com/connect.htm ). Linux uses techniques such as vdso, vsyscall, syscall to call kernel functions.

I find it fascinating to consider the design trade-offs of various C runtime libraries. With that in mind, here's a list of some of the C runtime library options:

eglibc:
http://www.eglibc.org/home
uclibc:
https://www.uclibc.org/
musl:
https://www.musl-libc.org/

Linux from Scratch build instructions (including musl)
http://clfs.org/view/clfs-embedded/arm/
midipix
http://midipix.org/
BSD regular expression library
(Musl regular expression support was forked from this library.)
https://github.com/laurikari/tre

PDCLib:
http://pdclib.e43.eu/
Original Public Domain C library:
https://sourceforge.net/projects/pdclib/
https://sourceforge.net/projects/pdos/
Another fork of PDCLib:
https://github.com/DevSolar/pdclib
libTom Public Domain libraries for math and cryptographics functions
(Some of musl's functions were forked from these.)
http://www.libtom.net/
libc11:
https://github.com/dryc/libc11
Eltanin-OS simia:
https://github.com/eltanin-os/simia

Bionic:
https://github.com/android/platform_bionic

Newlib:
https://sourceware.org/git/gitweb.cgi?p=newlib-cygwin.git;a=tree
libgw32c:
http://gnuwin32.sourceforge.net/packages/libgw32c.htm

Windows CE cross-compiler:
http://cegcc.sourceforge.net/
https://sourceforge.net/p/cegcc/code/HEAD/tree/

ELKS
https://github.com/jbruchon/elks

FUZIX
https://github.com/EtchedPixels/FUZIX

lib43:
https://github.com/lunixbochs/lib43

Embedded Artistry's libc:
https://github.com/embeddedartistry/libc

GNO
https://github.com/GnoConsortium/gno

Sortix C library:
https://gitlab.com/sortix/sortix/blob/master/libc/

MentOS:
https://github.com/mentos-team/MentOS/tree/master/libc/src

olibc:
https://github.com/olibc/olibc

CloudABI's standard C library:
https://github.com/NuxiNL/cloudlibc/

cc65 - C for 6502 systems:
https://github.com/cc65/cc65

kLIBC fork for OS/2:
https://github.com/bitwiseworks/libc

clib2 for Amiga
https://github.com/adtools/clib2

welibc:
https://bitbucket.org/wrelam/welibc/src/master/
I've listed some core utilities options besides GNU. I thought I'd share something about what utilities I personally prefer to use. My main requirement in a good set of core utilities is portability. This is rather hard to find. You would think that if a utility was efficient and lightweight, it would be easy to port. However, that's not necessarily so. Many utilities that are designed for efficiency take advantage of features of a particular operating system which makes them harder to port.

At first, I considered starting with sbase which had stated goals similar to what I was looking for, but it didn't have enough features to effectively replace the GNU core utilities when developing and building programs. While newer versions of sbase have added a lot of functionality, they've become much less portable.

My favorite source for inspiration is Minix. Earlier versions provided some interesting and fairly portable versions of a variety of utilites:
https://www.minix-vmd.org/cgi-bin/raw/source/std/1.7.5/src/commands/simple/
Some of the utilities don't have sufficient UTF-8 support or lack some newer functionality found in GNU utilities that makes them fail when attempting to build applications. However, they make a useful starting point.

In some cases, the OBase or BSD utilities do a better job than the older Minix ones and still do that job efficiently. I particularly like the version of patch found on BSD systems. It's an earlier variant of the Free Software Foundation's patch program. Unlike the FSF's version of patch which uses the GNU license, it uses a BSD style license.

For some utilities, I've consulted the POSIX standards ( http://pubs.opengroup.org/onlinepubs/9699919799/idx/utilities.html ) and rewritten them from scratch.

Rather than trying to port utilities such as the Free Software Foundations coreutils, I thought having a lightweight, efficient, highly portable option would be a useful alternative. Many of the FSF developers have little interest in portability making it hard to get later versions of their programs working on non-POSIX systems. I was surprised at how little interest most users have in developing portable alternatives to the core utilities that could be used to build software. Not only was their little interest, some people posted extremely negative comments when anyone suggested creating alternatives to the FSF software. I was also surprised by some of the negative reactions I read about wonderful projects like SBase.

I have a growing collection of public domain, BSD and MIT licensed alternatives to the GNU core utilities. For now, I just use them for my own projects. If you have an interest in portable utilities and tools, would like to see a viable portable alternative to the FSF's GNU coreutils or would like to further discuss related topics in a positive light, feel free to contact me. I'd enjoy talking with other developers and utility users on the topic.

You'll find some added information on my utilities and information on how to discuss the topic further at:
http://www.distasis.com/cpp/lmbld.htm
Most systems (other than BSD based ones) use GNU's core utilities. It's used by most Linux distributions. Cygwin, MinGW and gnuwin32 run ported versions of the GNU applications as well. Even Microsoft's SFU/SUA included some of the GNU utilities. However, the GNU core utilities are typically more bloated and have more feature creep than other versions of standard Unix utility programs. BSD systems have their versions of core utilities. The latest version of Minix has adopted the BSD utilities. They tend to be less bloated than the GNU versions, but are still more bloated than other options out there. The BSD utilities also tend toward adding new features similar to but not to the same extent as the GNU utilities. Also, some of their utilities aren't as well optimized as the GNU versions. Busybox seems like the most viable option for a lightweight but still comprehensive version of core utilities. I'm currently using it on my Debian system instead of the GNU core utilities. Toybox is a similar alternative to Busybox. It has a better license option than Busybox, but it's lacking some features and tools that Busybox has.

Here are some links to core utility collections:

Earlier Minix alternatives
http://www.minix-vmd.org/cgi-bin/raw/source/std/1.7.5/src/commands/simple/
Earlier versions of Minix put together an interesting collection of lightweight utilities from various sources.

Busybox
https://busybox.net/
Windows ports of Busybox:
https://frippery.org/busybox/
https://github.com/rmyorston/busybox-w32
https://github.com/realthunder/busybox-w32

Toybox
http://landley.net/toybox/

Heirloom Project
http://heirloom.sourceforge.net/
Based on traditional implementations of standard Unix utilities. Not very portable to non-POSIX systems. Not as bloated as GNU or BSD core utilities.

OBase
https://github.com/chneukirchen/obase
Port of OpenBSD userland to Linux.

SBase
http://git.suckless.org/sbase
This started out as a discussion on one of the suckless.org mailing lists of how to write efficient core utilites that weren't all part of one executable like Busybox or Toybox. Some good examples were posted and the project was started. Then project development was quiet for a while. The project became active again and one of the main goals besides efficiency was UTF-8/internationalization support. Looks like they've borrowed some UTF-8 support concepts (such as Runes) from Plan 9. It's not designed to be portable to non-POSIX systems. However, it does look like they've covered replacing most of the basic core utilities with lightweight, efficient versions.

Other alternatives:
https://github.com/jbruchon/elks/tree/master/elkscmd
https://github.com/EtchedPixels/FUZIX/tree/master/Applications
https://github.com/Orc/bin
https://github.com/eltanin-os/utilchest
https://github.com/eltanin-os/cbase
https://github.com/rofl0r/hardcore-utils
http://git.musl-libc.org/cgit/noxcuse/tree/
https://github.com/pikhq/pikhq-coreutils
http://www.fefe.de/embutils/
http://skarnet.org/software/s6-portable-utils/
https://github.com/dimkr/lazy-utils
https://github.com/arsv/minitools
http://git.suckless.org/ubase
https://github.com/dcantrell/bsdutils
https://github.com/cheusov/nbase
https://github.com/rswier/swieros
https://github.com/minoca/swiss
https://github.com/mentos-team/MentOS/tree/master/programs

April 2025

S M T W T F S
  12345
6789101112
13141516171819
20212223242526
27282930   

Syndicate

RSS Atom

Most Popular Tags

Style Credit

Expand Cut Tags

No cut tags
Page generated May. 23rd, 2025 01:51 pm
Powered by Dreamwidth Studios