Entries tagged with make

Entry tags:

Shell Environments for Windows

I remember sending in a bug report to the developers of GNU make. They refused to fix the bug because cygwin and msys bash were, in their opinion, the only viable shell options on Windows and the only environments they wanted to officially support. When I tried to build autoconf with a natively built version of make on Windows, it just hung. There's also the problem of using make on Windows without a shell. The command prompt has a limitation of how many characters can be passed to it. Typically, it's not enough to be able to run many of the commands used in standard makefiles. So, using make with a shell on Windows is a requirement to work with typical makefiles from many Free, Libre and/or Open Source software projects.

Several years later, the landscape for Windows has changed. It's still incredibly hard to find a standard port of POSIX shells outside of Cygwin, msys and now msys2 projects. However, there are options. Some of the older options that were around when I reported the bug are winbash (an older version of bash natively ported to Windows) and winZsh (which has not been maintained for a while). More recently, with the port of busybox to Windows in the busybox-w32 project, we have access to a modern shell. That project makes the ash shell available. It can run natively in Windows. The one drawback is that you need to build busybox-w32 on a Linux machine with a cross-compiler. I did recently manage to build ash natively on Windows with a lot of patching and my own custom makefile. Another option is to use midipix to build bash or other shells but it has similar licensing constraints to Cygwin and the msys projects. An alternative that I haven't seen a lot of information about is using bash from WSL via the Windows command prompt. You actually don't need msys or similar environments with this option. You can run a cross compiler and Linux tools via WSL and access them all from a standard Windows command prompt outside of WSL. That opens up a lot of possibilities. There were other projects that were supposed to port shells to Windows that still haven't shared much in the way of results. I remember offering to work with the developers of the mksh to port it to Windows. They said they already had someone working on the port. I've yet to see a publicly available, natively built mksh running on Windows. Plus, with all these Windows shell options, I still haven't seen a fix to the GNU make project. I do continue to patch my builds of make with my own patch for the issue.

WSL is great for development on Windows. Even with WSL, Busybox-w32 and environments like Cygwin, msys2, msys and midipix, I still like the idea of a minimal native build system and shell. I'd like something I can build from source for myself on Windows. I'd also prefer to use tools with more lenient Open Source licenses over GNU GPL and AGPL options. I'm going to be experimenting with replacing the msys environment on my system with more native options that don't obscure my actual file path names and don't require compatibility with GPL licenses to get POSIX style core utility programs to work. I will still need some standard POSIX style utilities and programs if I want to build FLOSS projects that use GNU autotools. However, I've been avoiding using cmake for FLOSS projects that use it by creating my own build files. I may start doing that in the case of some FLOSS projects that use GNU autotools. libressl comes to mind since a change to their build scripts made them no longer work on Windows and I've yet to hear if they'll fix the issue or not. I've been using cDetect, GNU make and pkgconf for my build scripts. I wish someone would come up with a simpler replacement for GNU make that still offered enough flexibility to build FLOSS projects that require make.

The build environment I'm working towards will probably look like the following... I'm building ash from Busybox-w32 natively with my own scripts. I'm using a combination of POSIX style core utilities from projects such as Minix, sbase and BSD. I'm hoping for compatibility with Open Group standards not necessarily with GNU standards for the various utilities. At the moment, I'm using the GNU gcc compiler built from source. However, llvm is a nice option. I just haven't seen an easy way to build it from source on Windows. My main goal is to be able to build all parts of my environment from source code. It may not have as many features or as much compatibility as other development environments available on Windows. However, I like the idea of being able to modify and change the source code of every tool I use whenever I want to. One other thing I'd like to look for is a console program that I can build natively with my C compiler. If I get everything working for Windows, a lot of it will probably port to other environments fairly easily.

I know most people prefer to just download a prebuilt environment or let their package manager do that for them and they don't care about the ability to build every part of their development environment from scratch. However, if this does sound interesting to anyone else, if you're working on something similar or if want to discuss parts of this project with me further, feel free to contact me. You can reach me via Mastodon or Bluesky.

Entry tags:

Native POSIX enviroment for Windows

While WSL is great, I've always wanted a native POSIX environment for developing on Windows. I wanted one that didn't require specialized DLLs with GNU GPL licenses such as as Cygwin and MSYS do. There are other options available. Midipix provides a POSIX environment for easily porting and developing POSIX compliant applications on Windows. The licensing isn't any more lenient than Cygwin though. It does use the MUSL C library which is a nice feature. There's AT&T's Uwin project which is now available as Open Source: https://github.com/att/uwin The license for the project isn't exactly compatible with GPL software. I've looked into native POSIX shell alternatives to those found in projects like Cygwin and msys. Bash is not easy to port to Windows without Cygwin or msys DLLs to support various functionality. It isn't exactly easy to port to DOS either. Most bash ports to DOS and Windows are older ones. One Windows port that builds successfully on Windows with MinGW32 is: http://win-bash.sourceforge.net/ I investigated the mksh project at one point and even offered to work on a Windows port. They were not in the least bit interested and already had some commercial company looking to create a port to Windows using there own proprietary code to provide a POSIX compatibility layer. So even if their plan was successful, it wouldn't exactly provide a native Windows port anyone could use in their projects. Tcsh has support for Windows. Zsh was also very popular on Windows. Checking the versions of zsh, the latest Windows version I could find that compiled with no issues using MinGW32 was 3.0.8.1: https://github.com/oldfaber/wzsh The zsh project seems to have progressed to a much later version since then. I've also experimented with Swiss which provides a Busybox-like set of tools with a shell. Speaking of Busybox, there is a port of it to Windows which provides an implementation of the ash shell: https://github.com/rmyorston/busybox-w32 Unfortunately, the build system works in such a way that you need to cross-compile it to get anything to build properly. Even after attempting to cross-compile using the source code, I still ended up with a program that crashes on exit. The busybox-w32 executables at the official sites don't seem to have that problem though. So much for being able to reproduce an executable others are distributing just from the source code.

I have several programs that could replace a lot of the core and basic utilities found on POSIX systems. I even have an older version of make patched to work with a shell of my choice rather than requiring Cygwin or msys DLLs or producing a crippled native version of make that can't handle long command lines. I had discussed the option of getting make to work with other shells besides Cygwin or msys with the developers of make at one point. I had sent them a patch for an issue with using make natively on Windows without a shell. The developers weren't interested in Windows support beyond using Cygwin or msys bash shells. I have older versions of GNU autotools patched to run natively and work around the slash versus backslash file delimiter issues. So, I think I could put together a native POSIX environment for Windows that could be used for development with a command line compiler. However, I'm still left with the decision of how to replace the msys bash shell. It would be very nice to avoid Cygwin and msys and not have the limitations on directories that these programs impose. For instance, they assume / and /usr are in the same locations so /bin is the same directory and in the same physical locaton as /usr/bin. A Windows native POSIX shell option would be ideal. The only question is which shell to go with.

Likely Windows shell candidates at this point are older versions of bash, zsh and ash. It would be nice to find some other options or hear from others interested in native Windows development as to what their POSIX shells of choice might be. If anyone else is interested in developing natively on Windows using a POSIX-like environment, feel free to contact me to discuss the situation further, add your suggestions or brainstorm some new ideas: http://www.distasis.com/connect.htm I'd love to be able to put something together similar to the gnuwin32 or unxutils projects but with more up-to-date utilities. It would be great to have input from others looking to create or piece together their own POSIX development environments on Windows.

Entry tags:

Preprocessor and Makefile Tips and Tricks

Here are some interesting articles and tips related to the C preprocessor and to creating makefiles.

This shows how to print the value of a macro:
http://c-faq.com/ansi/stringize.html

How to use static patterns with makefiles:
https://www.gnu.org/software/make/manual/make.html#Static-Pattern

Preprocessor tips:
The second article shows how to check which include files are used by a C or C++ file.

https://mischasan.wordpress.com/2013/04/07/the-c-preprocessor-not-as-cryptic-as-youd-think/
https://mischasan.wordpress.com/2011/06/14/stupid-gcc-trick-2-finding-all-included-files-recursively/
https://mischasan.wordpress.com/2011/10/12/stupid-gcc-trick-3-list-all-built-in-gcc-define-symbols/

Make tips:
The first article has a great tip for creating directories in makfiles using "%/..:".

https://mischasan.wordpress.com/2012/07/05/two-more-cheap-gmake-tricks-creating-directories-and-printing-variables/
https://mischasan.wordpress.com/2015/03/09/gmake-cheap-trick-3/
https://mischasan.wordpress.com/2015/03/21/gmake-cheap-trick-4-for-non-recursive-make/

To see what commands make will run when invoked, you can use:
make --just-print

Nonrecursive makefile tips:

https://mischasan.wordpress.com/2013/03/30/non-recursive-make-gmake-part-1-the-basic-gnumakefile-layouts/
https://mischasan.wordpress.com/2013/03/30/non-recursive-make-gmake-part-2-rules-mk/
https://mischasan.wordpress.com/2013/04/13/non-recursive-make-part-3-a-tool-for-the-fearless/

Replacing other utilities with sed:

In case you need these in a makefile, build script or elsewhere and don't have them on your system:

https://github.com/aureliojargas/sed.sf.net/blob/master/local/docs/emulating_unix.txt

Miscellaneous compiler tips:

To stop the gnu gcc compiler after the first error use:
-Wfatal-errors

To stop the gnu gcc compiler after the N errors use:
-fmax-errors=N
So for three errors add the following to the command line when invoking gcc:
-fmax-errors=3

My favorite preprocessor tips:

You can use a C preprocessor to generate templates.
Here's an example to generate web pages using the preprocessor and templates:
http://www.distasis.com/cpp/mingw.htm#makeprograms

For my build system, I need a more functionality than the standard C preprocessor could offer.
Rather than reinventing the wheel and writing my own preprocessor, I found gpp has just enough capabilities beyond a standard preprocessor to handle the task.
The gpp preprocessor is available from:
https://logological.org/gpp

Know of some other preprocessor or makefile tips? Have written an article or blog post with your tips on these topics? Please share them. They may get included in this list.

Entry tags:

Build Systems - Alternatives to GNU autotools, cmake, etc.

I've been looking for useful, efficient, user-friendly alternatives to build tools like GNU autotools and cmake. I've even created lists of build tool alternatives at various wikis and sites related to programming and compilers. If there's interest, I can post some of that information here along with pros and cons of various tools. I think, during my searches, I've found some good alternatives for my own projects and to use with my own build script generator.

If you're looking for build tools for your own projects, you may want to try the following tools. I'm also doing some active development and patching to some of the tools and would be interested in comparing notes with others on ways to use and improve them (see below for details).

GNU autotools is probably the most popular build system for most C/C++ programs. So, what's wrong with it? Just to build a C program, you need two interpreters (m4 and Perl). You also need autoconf, automake and make. The input to these programs is not necessarily intuitive to a C programmer. Many C programs, especially those using Qt make use of cmake which has less dependencies, but I've also found it less than intuitive. What if you want to build a system from scratch. You can get make to build without any other build tools. I've done it. From there, you can work to build the rest of the gnu autotools toolchain. To build cmake, you need to bootstrap the build using another instance of cmake. So where do you get the original build of cmake for the system? There is a way to build cmake with gnu autotools, but I've found it rather buggy.

After looking through several make alternatives, I really have not found a good replacement for it. The syntax takes some getting used to, but I've been able to find ways to get it do whatever I need it to do. There are situations where make is very inefficient, but there are work-arounds. Efficiency really depends on how well you craft your makefiles. I attempted to find some alternatives to the GNU version of make. BSD make has very different syntax, so only the simplest make scripts work with both tools. I find GNU make used more often than BSD make for building Open Source projects and it has features that were never added to BSD make. There's a Perl implementation of make, but that requires a Perl interpreter. My main issue with GNU make is that it is only supported on certain platforms. I sent in a bug report about GNU make not working properly on native Windows and I even sent in a patch. The developers were not interested in supporting native Windows. They thought it was enough that there were versions that worked with Cygwin and msys. Unfortunately, I've yet to find an alternative to make that handles the GNU makefile sysntax and doesn't require installation of an interpreter.

I've decided that until I can find anything better, I will continue to use GNU make. Since it is Free Software, I can modify the code to work on other systems that GNU make developers don't care to support. I can distribute the modified version if I distribute the source code. So, I use a modified version of GNU make that fixes platform specific bugs that I've encountered.

After searching through many alternatives, I've found a wonderful replacement to configure/autoconf. It's also more compact and I find it more intuitive than cmake. It's called CDetect. It's written in C and is comprised of only 3 files. The project is no longer actively developed. However, I've been making several modifications of my own to get it to suit my projects' needs. I've implemented some of the items on the original developers To Do lists. I'm still making modifications as I find features that are lacking, but it makes a useful replacement to configure. I've converted some of the Open Source projects I build from source regularly to use CDetect and custom makefiles instead of their build systems. If you're building from source, such as with a Linux from Scratch system, using CDetect means you don't need Perl, and several other programs installed just to build a C/C++ program.

Freedesktop.org advocates the use of pkg-config. It's used by many GNU projects as well. While I originally thought it was yet another complication when building from source, I've found pkg-config rather useful in simplifying build scripts. You don't have to worry about finding the right library, pkg-config can tell you where to find it. If you're cross-compiling, you can get have two sets of directories with pkg-config information, one for cross-compiled libraries and one for regular libraries. Change the path in an environment variable and your build script knows where to find the appropriate libraries. What I really didn't like about pkg-config was the complexity of the program and that it required a circular dependency with glib. I don't even want glib on my system at this point. Luckily, there is an alternative with friendly licensing called pkgconf. It's a drop in replacement for pkg-config. It's very helpful in finding where the right libraries are on a system. I've gone from never using pkg-config to use pkgconf in almost all my build scripts.

So, between CDetect, pkgconf and make, I now have all the tools I need to use with my build system in order to build the Open Source programs and libraries I use most in an automated fashion. The tools are all written in C, so they require no special interpreters or other languages just to build them.

If you'd like to try out CDetect with my patches or see some of the build scripts I'm using with these tools, check out the archive link on my LM BLD system page:
http://www.distasis.com/cpp/lmbld.htm

I've added several features to my patched version of CDetect. Check the notes.txt file for details. I'm currently making some new modifications and would love input from other developers/users. If you'd like to make suggestions or compare notes, please contact me through the CppDesign mailing list or my web site:
http://www.distasis.com/connect.htm