![[personal profile]](https://www.dreamwidth.org/img/silk/identity/user.png){kind=small}
pdcurses/ncurses applications
Apr. 20th, 2017 11:53 amI've covered SDL and FLTK based applications. Now I'd like to discuss pdcurses and ncurses based ones. ncurses is found on many POSIX systems especially Linux. pdcurses was used primarily on DOS and Windows systems. Support for X and SDL was later added. The X support uses some legacy functions, so it doesn't build well on modern Linux systems. However, the SDL support allows pdcurses to be ported to any system that supports the SDL library. That means it can run on systems like Syllable and Haiku not just POSIX systems with X Windows. ncurses worked only on POSIX systems for a long time, but more recently support was added for Windows and MinGW32.
So which is the better library to use? Depends on your needs. ncurses is very stable, provides good Unicode support on many operating systems and offers added functionality like panels, menus and forms. On POSIX systems, it runs in console mode, so you can run applications outside X Windows (or Wayland) or on systems with no X Windows installed. Applications built with ncurses can also be used via ssh and telnet. Unicode support for pdcurses is currently limited to UCS-2 (16 bits). It designed in such a way that expansion to UTF-8 or UTF-32 would mean major changes. It works in console mode, but on DOS/Windows only. You'll typically need X Windows if you run pdcurses on a POSIX systems. There are some work-arounds to this. You can use DirectFB or nano-x to run SDL in framebuffer and use pdcurses with the SDL backend. However, applications run in framebuffer mode aren't usable with ssh and telnet. You'll need something like vnc to remotely access them. There is also a pdcurses fork called win32a which provides native Windows support and better support for colors. pdcurses has some functions that are not typically standard with ncurses/curses such as specific mouse and clipboard functions. However, it lacks a lot of the extras that ncurses adds such as menu and form support. Both pdcurses and ncurses have functions for the standard ASCII character set and for wide characters (wchar). However, keep in mind that wchar is 16 bits on Windows (and DOS djgpp) systems and 32 bits on most POSIX systems (such as Linux and BSD). Personally, I prefer to use UTF-8 whenever possible for Unicode. I also prefer the new char32_t (or uint32_t if not available) in place of wchar_t in all my applications with internationalization support. However, pdcurses/ncurses aren't designed for these options. Of the two libraries, ncurses better supports characters in a Unicode character set beyond the initial 16 bits and is better designed to handle large character sets.
Even though ncurses has many advantages over pdcurses, I still like to work with pdcurses. It's a more compact library, with the backend code separated from the rest of the code to make it easy to port to other platforms. With this design and with its SDL support, it's easy to get pdcurses working on a wide variety of operating systems including some places ncurses has not been ported. As mentioned, libform and libmenu are not a part of pdcurses, but their functionality is available with ncurses. It turns out that other versions of curses did not originally have support for forms and menus and they were separate libraries. BSD still has libform and libmenu that work with its version of curses. I found it very easy to port libform and libmenu to work with pdcurses. There were some bugs in the libraries which I'm surprised no one has fixed yet, but I have patches for them. One other advantage of pdcurses built with SDL is that you can intermix SDL and pdcurses commands. pdcurses/ncurses are primarily console/text libraries. SDL offers graphics support. You can mix graphics from SDL with text from pdcurses and use pdcurses as a text user interface (GUI alternative) for SDL. That's a really nice option when developing SDL applications.
I've added SDL 2.x support to pdcurses, so it will port to systems that support SDL 1.x or SDL 2.x. pdcurses with the SDL backend, used bitmaps to represent characters. That curtailed Unicode support. I've added support to use SDL_TTF for drawing characters. Now the only limitation on characters that can be used with PDCurses is the 16 bit limit to the character size built into the system. It should be possible to expand that to 32 but it would require modifications that would affect the entire design. Also, since curses only supplies ASCII and wchar support for routines, a Windows (or DOS) system would still be limited to 16 bits. One would need a new set of routines with UTF-32 support or UTF-8 support. The current routines could not be used to support 32 bit characters on systems where wchar is limited to 16 bits. The ASCII functions are designed to work with one character at a time not several which UTF-8 might require.
Some of my patches for better internationalization support for pdcurses have been added back in to the official version of pdcurses. All of my pdcurses patches plus documentation on them and my build scripts can be found by following the archive link at http://www.distasis.com/cpp/lmbld.htm I'm interested in adding some of the win32a patches and continuing to improve support for internationalization in the future. If anyone wants to discuss design ideas for continuing to add better Unicode support for pdcurses, please contact me.
The good news with ncurses/pdcurses based applications is that it's relatively easy to port them from one version of curses to another in many cases. It may require changes in the names of header files. Non-portable functions such as those supporting a mouse may not be available on some systems. However, one can typically use any application designed for a particular version of curses with another version of curses. That does not mean ncurses/pdcurses applications can easily be ported to other operating systems. For instance, if an application uses functions that aren't very portable across platforms (such as fork which isn't easy to emulate on Windows and doesn't work on DOS), it will be very difficult to port that application. Unfortunately, many ncurses applications do just that and use POSIX functions that aren't widely ported to non-POSIX systems.
So, let's talk about some of the applications that I've found that do port across a variety of operating systems and can be built with pdcurses.
There are several ncurses based hex editors available. I personally like the interface for ncurses hexedit. It reminds me of the kzap program I used to use with DOS, so I find it user-friendly. shed is another interesting option. It handles large file sizes extremely well.
You can find the programs here:
http://www.rogoyski.com/adam/programs/hexedit/onlinedoc/hexedit.html
https://sourceforge.net/projects/shed/
I have patches/build scripts for ncurses hexedit to work with pdcurses and port to other platforms. I've built shed on Windows before, but haven't created my own build scripts/patch set for it yet.
There aren't a lot of music applications that work well in console mode and provide a text user interface, but there are some. The ones that come to mind are TiMidity++ and GramoFile. Make sure if you intend to work with TiMidity++ that you start with the source from version control not the very old, outdated tarball at Sourceforge. Many features, including better Karaoke midi support, have been added since the tarball was created. I find it discouraging that distributions such as Debian refuse to update to the latest supported version of this project. I always build TiMidity++ from source myself to make sure I have the updates I need. I'm currently working on a build script for TiMidity++ that will allow me to build it as a library so it can be used with other applications. I currently have build scripts for TiMidity++ and freepats (Open Source sound fonts used with TiMidity++) and gramofile.
https://sourceforge.net/projects/timidity/
http://www.opensourcepartners.nl/~costar/gramofile/
I originally looked for library versions of libform and libmenu because I thought a lot of curses applications would make use of these features. Turns out, not as many use it as I originally estimated. One program that makes use of the functionality for these libraries is ckpass. It's a console based password keeper that is compatible with KeePass 1.x format. That means, if you want a lightweight alternative to the earlier versions of KeePass or KeePassX, you can still have console access to your passwords. The program looks great and it's easy to use. Unfortunately, it offers just read-only support for viewing password information. There's no easy way to modify password information using the program. However, it isn't that hard to add code to allow editing as well as viewing of passwords using libform. I'm working on some modifications to ckpass to give it more functionality, so it will become a useful, more lightweight replacement to other password storage alternatives. There are other command line based password storage options available and some even use portable formats so that you can find cross-platform solutions. However, I like the idea that ckpass supports the KeePass 1.x format which many other GUI password programs also use. This is the link to the original ckpass project:
https://sourceforge.net/projects/ckpass/
The final category I'll cover is the text editor category. There are a great many text editors that work in console mode or build with ncurses as can be seen from the listings at TextEditors.org ( http://texteditors.org/cgi-bin/wiki.pl ). However, the large majority of them are not highly portable.
When the competition between vi and emacs comes up and people start debating what should be the default editor on a Unix or Linux system, I typically skip both and go with nano. nano was based on the pico text editor. It wasn't a fork. It was coded from scratch, so there would be no issues with code licenses. pico was the main editor for the alpine mail system. Compared to other editors with unusual key sequences that you need to memorize to get anything to work, I find it very user-friendly. It has a menu and easy access to help if you forget what command keys you need to perform a specific function. My personal preference is to use a programming editor that will allow you to map key commands however you see fit. That avoids large learning curves if you didn't originally use vi or emacs as your first text editor and memorize their commands. However, if a text editor with keymapping support is not available, a user-friendly text editor with easy to access command menus and help descriptions is a good second choice. Most POSIX systems I work with have either pico or nano available or they can be installed fairly easily. So, if I need to do a quick edit from the console on a POSIX system, my default editor is typically nano or pico. One major drawback to pico is that it only handles one file at a time. You cannot have multiple files open and flip between editing them or cut and paste between them. nano adds support for multiple files, but the commands to switch aren't particular easy to remember.
I did a port of nano to Windows several years ago. There's a lite way to build nano (tiny) and a way to build it with more functionality. Building it the lite way and adding some modifications for porting, I was able to compile it successfully on Windows. So, while it didn't provide all the functionality I used on POSIX systems, it did work on Windows. A few years ago, I found a Windows port of nano on github. It didn't provide extra functionality, but it was interesting to see how the author went about porting the application and the differences between my port and that one. According to the nano editor FAQ, it now looks like there's some official support for Win32. However, when I tried to build the latest version, I still ran across a lot of the same issues as when I built previous versions. Only the lite version is supported for Windows. There were some assumptions about MinGW types that aren't true for all versions of MinGW. Rather than using configure for what it was designed for and testing if there needs to be a work-around for a particular system, the code has ifdefs that assume MinGW always handles things wrong and tries to fix the situation. In the process, it causes errors in the build for MinGW systems that do handle the situation properly. Guess I'll need to stick with my port or a Windows specific port like the one at github if I want nano to build properly on Windows.
nano is a great lightweight alternative for a text editor that works on a large variety of operating systems. When I need to modify a file and want an editor to quickly call from the command line, it is often my editor of choice. You'll find the source code and more information on nano at:
https://www.nano-editor.org/
As I mentioned, I prefer programming editors that have good support for key-mapping. The scintilla text editing component offers just that. A large variety of Open Source editors, including SciTE (my favorite GUI editor) and fxite (lightweight FOX Toolkit based GUI editor), use the scintilla text editing component. While fxite is lightweight and works well on older systems, it requires X Windows on POSIX systems and the FOX Toolkit to build it from source. I wanted something even more lightweight. I did a search of the various editors that make use of the scintilla text editing component and I ran across textadept. It is a GUI text editor, but there's a ncurses based version that works from the console. I found it relatively easily to build the editor on Windows using pdcurses. So, it's rather portable. If I had to pick a console based editor besides nano and pico, textadept would be it.
https://foicica.com/textadept/
There are ncurses front ends to some applications I work with such as hunspell and prozilla, but I typically don't use the front ends. I prefer using hunspell via command line or integrated with a programming editor. There's a nice FLTK front end for prozilla that is highly portable. One other ncurses based application I'll mention is WordGrinder. I typically see it mentioned on sites and blogs that specialize in discussing console based applications. It has many of the features of a word processor. The current web site for it is:
http://cowlark.com/wordgrinder/
I haven't tried it out, but nanotodon looks like a useful cross-platform client for Mastodon. Was able to get it to build on Windows using pdcurses.
https://github.com/taka-tuos/nanotodon
If you have other recommendations for useful, cross-platform portable ncurses/pdcurses/curses based applications, I'd love to hear ( http://www.distasis.com/connect.htm ) about them. It would be great to add more helpful applications to this list. I'm also very interested in the possibilities of using pdcurses/libmenu/libform as a text user interface (GUI alternative) for SDL applications.
So which is the better library to use? Depends on your needs. ncurses is very stable, provides good Unicode support on many operating systems and offers added functionality like panels, menus and forms. On POSIX systems, it runs in console mode, so you can run applications outside X Windows (or Wayland) or on systems with no X Windows installed. Applications built with ncurses can also be used via ssh and telnet. Unicode support for pdcurses is currently limited to UCS-2 (16 bits). It designed in such a way that expansion to UTF-8 or UTF-32 would mean major changes. It works in console mode, but on DOS/Windows only. You'll typically need X Windows if you run pdcurses on a POSIX systems. There are some work-arounds to this. You can use DirectFB or nano-x to run SDL in framebuffer and use pdcurses with the SDL backend. However, applications run in framebuffer mode aren't usable with ssh and telnet. You'll need something like vnc to remotely access them. There is also a pdcurses fork called win32a which provides native Windows support and better support for colors. pdcurses has some functions that are not typically standard with ncurses/curses such as specific mouse and clipboard functions. However, it lacks a lot of the extras that ncurses adds such as menu and form support. Both pdcurses and ncurses have functions for the standard ASCII character set and for wide characters (wchar). However, keep in mind that wchar is 16 bits on Windows (and DOS djgpp) systems and 32 bits on most POSIX systems (such as Linux and BSD). Personally, I prefer to use UTF-8 whenever possible for Unicode. I also prefer the new char32_t (or uint32_t if not available) in place of wchar_t in all my applications with internationalization support. However, pdcurses/ncurses aren't designed for these options. Of the two libraries, ncurses better supports characters in a Unicode character set beyond the initial 16 bits and is better designed to handle large character sets.
Even though ncurses has many advantages over pdcurses, I still like to work with pdcurses. It's a more compact library, with the backend code separated from the rest of the code to make it easy to port to other platforms. With this design and with its SDL support, it's easy to get pdcurses working on a wide variety of operating systems including some places ncurses has not been ported. As mentioned, libform and libmenu are not a part of pdcurses, but their functionality is available with ncurses. It turns out that other versions of curses did not originally have support for forms and menus and they were separate libraries. BSD still has libform and libmenu that work with its version of curses. I found it very easy to port libform and libmenu to work with pdcurses. There were some bugs in the libraries which I'm surprised no one has fixed yet, but I have patches for them. One other advantage of pdcurses built with SDL is that you can intermix SDL and pdcurses commands. pdcurses/ncurses are primarily console/text libraries. SDL offers graphics support. You can mix graphics from SDL with text from pdcurses and use pdcurses as a text user interface (GUI alternative) for SDL. That's a really nice option when developing SDL applications.
I've added SDL 2.x support to pdcurses, so it will port to systems that support SDL 1.x or SDL 2.x. pdcurses with the SDL backend, used bitmaps to represent characters. That curtailed Unicode support. I've added support to use SDL_TTF for drawing characters. Now the only limitation on characters that can be used with PDCurses is the 16 bit limit to the character size built into the system. It should be possible to expand that to 32 but it would require modifications that would affect the entire design. Also, since curses only supplies ASCII and wchar support for routines, a Windows (or DOS) system would still be limited to 16 bits. One would need a new set of routines with UTF-32 support or UTF-8 support. The current routines could not be used to support 32 bit characters on systems where wchar is limited to 16 bits. The ASCII functions are designed to work with one character at a time not several which UTF-8 might require.
Some of my patches for better internationalization support for pdcurses have been added back in to the official version of pdcurses. All of my pdcurses patches plus documentation on them and my build scripts can be found by following the archive link at http://www.distasis.com/cpp/lmbld.htm I'm interested in adding some of the win32a patches and continuing to improve support for internationalization in the future. If anyone wants to discuss design ideas for continuing to add better Unicode support for pdcurses, please contact me.
The good news with ncurses/pdcurses based applications is that it's relatively easy to port them from one version of curses to another in many cases. It may require changes in the names of header files. Non-portable functions such as those supporting a mouse may not be available on some systems. However, one can typically use any application designed for a particular version of curses with another version of curses. That does not mean ncurses/pdcurses applications can easily be ported to other operating systems. For instance, if an application uses functions that aren't very portable across platforms (such as fork which isn't easy to emulate on Windows and doesn't work on DOS), it will be very difficult to port that application. Unfortunately, many ncurses applications do just that and use POSIX functions that aren't widely ported to non-POSIX systems.
So, let's talk about some of the applications that I've found that do port across a variety of operating systems and can be built with pdcurses.
There are several ncurses based hex editors available. I personally like the interface for ncurses hexedit. It reminds me of the kzap program I used to use with DOS, so I find it user-friendly. shed is another interesting option. It handles large file sizes extremely well.
You can find the programs here:
http://www.rogoyski.com/adam/programs/hexedit/onlinedoc/hexedit.html
https://sourceforge.net/projects/shed/
I have patches/build scripts for ncurses hexedit to work with pdcurses and port to other platforms. I've built shed on Windows before, but haven't created my own build scripts/patch set for it yet.
There aren't a lot of music applications that work well in console mode and provide a text user interface, but there are some. The ones that come to mind are TiMidity++ and GramoFile. Make sure if you intend to work with TiMidity++ that you start with the source from version control not the very old, outdated tarball at Sourceforge. Many features, including better Karaoke midi support, have been added since the tarball was created. I find it discouraging that distributions such as Debian refuse to update to the latest supported version of this project. I always build TiMidity++ from source myself to make sure I have the updates I need. I'm currently working on a build script for TiMidity++ that will allow me to build it as a library so it can be used with other applications. I currently have build scripts for TiMidity++ and freepats (Open Source sound fonts used with TiMidity++) and gramofile.
https://sourceforge.net/projects/timidity/
http://www.opensourcepartners.nl/~costar/gramofile/
I originally looked for library versions of libform and libmenu because I thought a lot of curses applications would make use of these features. Turns out, not as many use it as I originally estimated. One program that makes use of the functionality for these libraries is ckpass. It's a console based password keeper that is compatible with KeePass 1.x format. That means, if you want a lightweight alternative to the earlier versions of KeePass or KeePassX, you can still have console access to your passwords. The program looks great and it's easy to use. Unfortunately, it offers just read-only support for viewing password information. There's no easy way to modify password information using the program. However, it isn't that hard to add code to allow editing as well as viewing of passwords using libform. I'm working on some modifications to ckpass to give it more functionality, so it will become a useful, more lightweight replacement to other password storage alternatives. There are other command line based password storage options available and some even use portable formats so that you can find cross-platform solutions. However, I like the idea that ckpass supports the KeePass 1.x format which many other GUI password programs also use. This is the link to the original ckpass project:
https://sourceforge.net/projects/ckpass/
The final category I'll cover is the text editor category. There are a great many text editors that work in console mode or build with ncurses as can be seen from the listings at TextEditors.org ( http://texteditors.org/cgi-bin/wiki.pl ). However, the large majority of them are not highly portable.
When the competition between vi and emacs comes up and people start debating what should be the default editor on a Unix or Linux system, I typically skip both and go with nano. nano was based on the pico text editor. It wasn't a fork. It was coded from scratch, so there would be no issues with code licenses. pico was the main editor for the alpine mail system. Compared to other editors with unusual key sequences that you need to memorize to get anything to work, I find it very user-friendly. It has a menu and easy access to help if you forget what command keys you need to perform a specific function. My personal preference is to use a programming editor that will allow you to map key commands however you see fit. That avoids large learning curves if you didn't originally use vi or emacs as your first text editor and memorize their commands. However, if a text editor with keymapping support is not available, a user-friendly text editor with easy to access command menus and help descriptions is a good second choice. Most POSIX systems I work with have either pico or nano available or they can be installed fairly easily. So, if I need to do a quick edit from the console on a POSIX system, my default editor is typically nano or pico. One major drawback to pico is that it only handles one file at a time. You cannot have multiple files open and flip between editing them or cut and paste between them. nano adds support for multiple files, but the commands to switch aren't particular easy to remember.
I did a port of nano to Windows several years ago. There's a lite way to build nano (tiny) and a way to build it with more functionality. Building it the lite way and adding some modifications for porting, I was able to compile it successfully on Windows. So, while it didn't provide all the functionality I used on POSIX systems, it did work on Windows. A few years ago, I found a Windows port of nano on github. It didn't provide extra functionality, but it was interesting to see how the author went about porting the application and the differences between my port and that one. According to the nano editor FAQ, it now looks like there's some official support for Win32. However, when I tried to build the latest version, I still ran across a lot of the same issues as when I built previous versions. Only the lite version is supported for Windows. There were some assumptions about MinGW types that aren't true for all versions of MinGW. Rather than using configure for what it was designed for and testing if there needs to be a work-around for a particular system, the code has ifdefs that assume MinGW always handles things wrong and tries to fix the situation. In the process, it causes errors in the build for MinGW systems that do handle the situation properly. Guess I'll need to stick with my port or a Windows specific port like the one at github if I want nano to build properly on Windows.
nano is a great lightweight alternative for a text editor that works on a large variety of operating systems. When I need to modify a file and want an editor to quickly call from the command line, it is often my editor of choice. You'll find the source code and more information on nano at:
https://www.nano-editor.org/
As I mentioned, I prefer programming editors that have good support for key-mapping. The scintilla text editing component offers just that. A large variety of Open Source editors, including SciTE (my favorite GUI editor) and fxite (lightweight FOX Toolkit based GUI editor), use the scintilla text editing component. While fxite is lightweight and works well on older systems, it requires X Windows on POSIX systems and the FOX Toolkit to build it from source. I wanted something even more lightweight. I did a search of the various editors that make use of the scintilla text editing component and I ran across textadept. It is a GUI text editor, but there's a ncurses based version that works from the console. I found it relatively easily to build the editor on Windows using pdcurses. So, it's rather portable. If I had to pick a console based editor besides nano and pico, textadept would be it.
https://foicica.com/textadept/
There are ncurses front ends to some applications I work with such as hunspell and prozilla, but I typically don't use the front ends. I prefer using hunspell via command line or integrated with a programming editor. There's a nice FLTK front end for prozilla that is highly portable. One other ncurses based application I'll mention is WordGrinder. I typically see it mentioned on sites and blogs that specialize in discussing console based applications. It has many of the features of a word processor. The current web site for it is:
http://cowlark.com/wordgrinder/
I haven't tried it out, but nanotodon looks like a useful cross-platform client for Mastodon. Was able to get it to build on Windows using pdcurses.
https://github.com/taka-tuos/nanotodon
If you have other recommendations for useful, cross-platform portable ncurses/pdcurses/curses based applications, I'd love to hear ( http://www.distasis.com/connect.htm ) about them. It would be great to add more helpful applications to this list. I'm also very interested in the possibilities of using pdcurses/libmenu/libform as a text user interface (GUI alternative) for SDL applications.
![[community profile]](https://www.dreamwidth.org/img/silk/identity/community.png){kind=small}