Changelog for Spread releases

The current version of Spread can be downloaded off our download page.

Spread 3.17.3

The 3.17.3 release includes a few enhancements and a number of fixes. They include fixing some daemon crashes, a substantial decrease in memory usage under some scenarios, new build files for VC++.NET, a Spread sptmonitor program that works on Windows, and some small cleanups and stability improvements. So we highly encourage everyone to upgrade to this release.

The 3.17.3 release has no external api changes (but does fix a few api implementation bugs so check the details below).

New minor features:
1) Add new option to spflooder. The -n option allows a fixed "number of members" to be set. This then activates a multi-sender flow control algorithm to allow flooding tests with several senders. All of the processes need to join the group (i.e. they cannot be -wo (write-only)) but not all processes have to send. This allows easy testing with differing numbers of senders (just change how many spflooders start with -ro and how many do not).
2) Modify monitor so it will run correctly on Windows. This was done by modifying monitor.c so it can also build as a threaded program and on windows will use multiple threads to send/recv updates and get user input. (A POSIX thread option is also added).
3) Add new Alarm priority flag to print a line with no datestamp (for multi-line output).
4) Add new Windows VC++.Net project files to daemon/win32_msvc_net. The current project files in daemon/win32 remain as they can be used in VC++ (version 6).

The list of bugfixes is:
1) Fix memory leak in Skiplist. Reported by Taj Khattra, patch by Theo Schlossnagle. 2) spuser,spflooder, and spmonitor fixed to print correct name in help. Patch by Daniel Rall.
3) Fix incorrect alarm printing where WARNING messages generated by older Alarm() interface were not printed.
4) Fix a bug in the Windows build using VC++ of thread-safe libtspread.lib. Bug and fix by Jacob Green.
5) Fix bug in libspread where if the groups array or message body passed to SP_recv* was too small, the mess_type field returned would be truncated and the sender field was not returned. They are both now returned correctly. Bug report and partial fix provided by John Schultz.
6) Fix bug where SP_Join and SP_Leave do not report an error if a group name is too long (instead they truncated it) Reported with fix by David Parker.
** Warning, this could break buggy applications who use long groups and assume the name is truncated.
7) Cleanup compile warnings where E_queue() used with no-parameter functions (not all uses fixed) and fix incorrect use of signed int with strlen().
8) Fix few cases in flooder.c and user.c that did not use the defined MAX_MESSLEN constant. Tested to verify that increasing MAX_SCATTER_ELEMENTS in scatter.h and the MAX_MESSLEN defines in user.c and flooder.c is sufficient to support arbitrarily large message sizes with Spread. This is NOT recommended, but several people do it anyway :-)
9) Make E_delay() work on Win32.
10) Added check that a segment using localhost (i.e. 127.0.0.255) is NOT in a spread.conf with any other segments using real IP addresses. This configuration will not work correctly and is usually because the example localhost configuration provided with Spread is added to. This check will cause the daemon to exit immediately after parsing the config file.
11) Fix bug where if more then 22 daemons start at the same time, some will crash or the membership will not complete correctly. This bug was reported by several people including Jesse Noller.
12) Fix Java spread connection problem where several threaded connections from same process cause unexpected connection failures. Reported by Brian Moseley, idea of fix by Ryan Caudy, patch by Jonathan Stanton.
13) Fix EVS bug where AGREED messages may be delivered before a transitional signal on some daemons and after it on others. Bug found and patch created by Ryan Caudy.
14) Make SP_connect_timeout() calls non-blocking for the actual 'connect()' call. This should fix the issue reported by Shlomi Yaakobovich where a hung daemon causes new connections to also hang in connect. Also includes slight cleanup of connect code path.

Spread 3.17.2

This release includes a number of bugfixes, including some that fix daemon crashes, a decrease in message token overhead, and some small cleanups and stability improvements. So we highly encourage everyone to upgrade to this release.

The 3.17.2 release has no new features or external api changes.

One thing to note is the parsing of the spread.conf file has become more strict, and some config files that would parse without error will now cause Spread to fail to start and report a configuration file error. Since all of these cases could cause the daemon to misbehave or crash at runtime (but not at startup) this change increases stability by checking for misconfigured setups before they cause unpredictable behavior.

• Fix daemon quit when multiple interfaces are configured as "D" daemon interfaces in the spread.conf file. Bug reported by Orit Wasserman.
• Updated url for Java 'ant' build system. Patch by Daniel Rall.
• Fix group_id bug that causes incorrect vs_sets. Patch by Ryan Caudy.
• Fix spread.conf parser so it validates the machine names in segments and forces them to be less then MAX_PROC_NAME. Patch by Mikhail Terekhov.
• Minor fix to Mac OS X compilation so library softlinks do not fail the second time make is run.
• Alarm() changes to support priority levels on each Alarm() call.
• Fix crash by improving packet accounting when a client connected to a singleton daemon sends a large broadcast. Reported by David Shaw.
• Fix bus errors on Sparc & Alpha for message buffer integer assignment. Reported by Greg Shebert; tested and patched Mikhail Terekhov.
• Verify daemon names in spread.conf are unique. If non-unique names are provided in spread.conf, configuration will be rejected and daemon will not start. Suggested by Tim Peters.
• Zero buffer in c library before sending multicast. Reported by Panagiotis Kougiouris.
• Send fewer lookup probe messages when only a single segment is configured.
• Remove extra token rotations when no messages are sent. Will decrease network packet overhead.
• Make mailbox and service in sp.h a typedef instead of a #define. Suggested and patched by Steven Dake.
• Fix small endianness error in sp.c where the mess_type field may not be correctly converted for different endian platforms when the SP_*_recv calls return a BUFFER_TOO_SHORT or GROUPS_TOO_SHORT error.
• Change alarm tag for security prints from SEC to SECURITY because of conflict with sys/time.h header.
• Documentation fix to SP_receive man page to correct fields for self-leave membership messages.
• Update of email addresses in copyright statements and headers.
• Windows binary libraries are now built as libspread and libtspread like other platforms.

Version 3.17.1

This release includes a number of bugfixes, including some that fix daemon crashes and potential security issues, and some small cleanups and stability improvements. So we highly encourage everyone to upgrade to this release.

The 3.17.1 release has no new features, api changes or other dramatic changes. The potential security issue is a buffer overflow in the C language CLIENT library that could be exploited by a malicious daemon or man-in-the-middle attack to execute code with the privileges of the user running the client. This bug was uncovered by a DARPA funded Red Team from SRI who were evaluating Spread and Secure Spread.

• Fix memory corruption and crash with groups of large size.
• Correct make install so it installs header files.
• Fix syntax error in build.xml file for Java/Ant.
• Cleanup prototypes to remove compiler warnings.
• Fix parser to correctly recognize upper, lower, and mixed case command options.
• During make install, remove old symlinks.
• Change setgroups call to be more portable. (fixes MacOSX)
• Change name of r and s to sprecv and spsend, and add as make targets. They can be built by "make testprog" (not built by default).
• Work on making long group names possible.
• Increase listen backlog for accepting client connections.
• Fix Win32 project files to have correct path to source files. (note CVS was always ok, but 3.17.0 release had incorrect path)
• Fix bug where large groups overflow Mess_buf in groups.c.
• Fix memory corruption bug when a message header is received in several separate packets in session.c. Thanks to Ryan Caudy for many, many hours tracking this down.
• Change order of build in Makefile so binaries are built before documentation.
• Fix Java bug where connection objects cannot be disconnected and then reconnected, but must be created anew. They can now be reused.
• Fix compile error on AIX for struct if_info.
• Fix security issue with buffer checks in the C library.
• Fix obscure off-by-one buffer error with the parser.

Version 3.17.0

This is a major release including new features, non-backwards compatible changes in the protocols and build environment, performance improvements and bugfixes.

• Enhance the group membership algorithm so very large groups and very large numbers of groups will be merged correctly during a membership. With this change Spread should correctly handle groups with thousands of members and thousands of groups. This also includes the optimization of sending groups messages as FIFO and AGREED instead of SAFE. Thanks to Ryan Caudy from Hopkins for doing this work.
• Revamped build setup using autoconf instead of manual Makefiles. Thanks to Theo Schlossnagle for writing this and updating it. Some small changes were made to Theo's 3.16.2 patch to incorporate it.
• Applied Daniel Rall's SO_REUSEADDR patch. This adds a new configuration option SocketPortReuse that can either force SO_REUSEADDR on or off for TCP server sockets, or set it to an auto mode that uses REUSEADDR if specific interfaces are specified in the spread.conf file and does not use it if the default of INADDR_ANY is used. If you force this on, be aware that it can open up a security risk where other processes can steal Spread's traffic.
• Changed library name from libsp to libspread. This is both more descriptive and should fix the problem with library clashes with other libsp's.
• Applied Daniel Rall's patch to make the spread execution directory and the user and group it runs as configurable at runtime.
• Fix GGT bug where Spread gets stuck or crashes if you reach GGT state. This bug was introduced when Spread 3.x adopted the research version of the session layer.
• Make sure service_type is set to 0 before using it in SP_receive calls in sample programs (spuser, spflooder, simple_user).
• Fix Mem_Alloc blocks so they can be freed without assertion failure.
• Fix hanging bug where token rotates, but ARU is stuck.
• Split the growing changelog at the beginning of Readme.txt into a separate file.
• Updates to man pages. Clarify service_type handling. Add info on accessor functions for membership message body.

Version 3.16.2

This is a major bugfix release. The focus was stability, bugfixes and some documentation.

• All recv calls in session now check for EWOULDBLOCK, EAGAIN, EINTR and retry the call later. This fixes problem on busy servers where connections would be killed erroneously. Thanks to the Zope team (Guido van Rossum, Tim Peters, and Jeremy Hylton) for discovering this and helping with fixing it.
• log.c file operations now check for errors and handle them.
• Windows bugs:
  • close() does not close sockets, we need to call closesocket() to avoid memory leak.
  • sockets are not consecutive fd's on windows, they can be > FD_SETSIZE. So fd mapping to sessions changed to use a hash instead of array. As a side-effect it should be faster to connect/disconnect. Thanks to Marc Zyngier for discovering the problem and writing and testing a patch for it.
  • Raise the FD_SETSIZE for windows to 1024 by #defining it in arch.h.
  • Enhanced error handling to support WSAGetLastError, and define a sock_strerror() function to return nice error strings on windows.
  • Include Visual C++ Project files to build Spread on Windows.
• Included improved NetBSD makefile and warning fix from Patrick Welche.
• Generalize the errno handling to support windows errors and other odd cases. Also see fix 3d. Thanks to Marc Zyngier, Tim Peters, and David Holmer for help with this.
• Improve the install targets for linux. Now install include headers and use standard PREFIX define for base directory.
• Updates to man pages. New libsp manpage.

Version 3.16.1

This is a major bugfix release. The focus was stability, bugfixes and a few new platform makefiles.

• Fix various races and bugs in the new authentication and access control code.
  • Fix session establishment races with authentication
  • Add more info for access control hooks to use (mess_type and num_groups)
  • Fix so rejected messages are not still delivered to local group members.
  • Make reject messages honor SELF_DISCARD.
  • Fix bug that always allowed NULL connections.
• SGI now builds using signed chars and reports errors correctly.
• MacOSX build fixed.
• Fixed race at session establishment that caused the assertion failure "assert( xxxxx ) false "
• Add MAX_MESSAGE_BODY_LEN to spread_params.h (used in sp.c checks), move MAX_PACKET_SIZE to spread_params.h from arch.h.
• Fixed interface restrictions for daemons.
• Fixed crashes in user and flooder programs when wrong number of command line parameters were passed.
• Added assertion check for Groups membership message with zero members in a group. (Theo)
• Java library fixes and enhancements (Thanks to Dan Rall and Leonard Richardson )
• Fix bug in group name validator G_private_to_names that caused crash.
• Fix for reversed parameters to memset in groups.c (Thanks to Marc Zyngier)
• Fix bug in session connection that confused closed, but not yet deleted file descriptors with newly accepted connections. (Thanks to Dave Parker for discovering bug).
• Fix significant memory leak when connections are disconnected. This leak has been present since at least 3.15.0. (Thanks to Dave Parker for discovering this and providing lots of assistance debugging it.)
• Fix the binding to specific interfaces to work with broadcast packets. (Thanks to Marc Zyngier for reporting the problem)
• Updated man pages and Users Guide
• Added makefile for OpenBSD contributed by Brandon Palmer.
• Added makefile for Solaris 8 with Solaris cc compiler 6.1 contributed by Luc I. Suryo.

Version 3.16.0

This is a major release. The main change is a new open-source license. This release also includes several new features such as support for multiple interfaces on daemon machines and a new access control and authentication framework. It includes significant performance improvement in joins and leaves and lots of smaller changes:

• Add support for configuring multiple interfaces on daemons. Each daemon can now choose which interfaces to listen on for client connections, monitor requests, and daemon-to-daemon communication.
• Added support for MacOSX. Thanks to Julien Defour and Dirk-Willem van Gulik.
• Agreed messages for join/leave instead of SAFE messages.
• Fixed bug with membership messages for more then 42 members in a group.
• Document how to monitor packing effectiveness.
• SP_connect assigns unique private name if user gives a zero length string as the user name.
• Slightly loosen valid field checking for service_type field to allow flush and secure spread libraries to work.
• Fix events to work with low priority events for Splash.
• Adds modular access control and authentication system that allows the daemon to authenticate client connections and allow or disallow various actions (such as joining groups, sending messages or connecting) based on the authenticated user. This changes the client-server network protocol, but is backward compatible with clients linked with old libraries. This also adds new functions to the API. A sample module that provides access control based on the IP address of the client is included. Also a module that provides the same "allow all clients" policy as older versions of Spread. The Java API has also been updated to support the new access control framework.
• New license for Spread. BSD based open-source license.
• Updated Perl library with some bug fixes and compatibility with new C lib.
• Added support for 64 bit platforms (alpha, ultrasparc & ia64) Note only alpha is actually tested.
• Fix bug in sp.c where if a message was not completely received because of a short buffer and the next call had an invalid message scatter the data in the messages would get corrupted. Probably difficult to trigger unless you are trying hard.
• Changed names of programs included with spread to have "sp" as a prefix. This should help avoid conflicts with all the other utilities called "user" or "monitor".
• Small bugfixes to the Java library.
• Bugfix which removes a costly, and unnecessary, recalculation when several network level membership events occur in close (time) proximity.

Version 3.15.2

There are no code changes to the main spread daemon or C library.

The java bug was the one preventing connections to the daemon in SpreadConnection class.

The perl bug was a memory leak.

The way to test if the SP_version function is the old (3.14-) or new (3.15.1+) interface is to do:

#ifdef SPREAD_VERSION
 /* use new interface */
#else
 /* use old interface */
#endif

Version 3.15.1

This release had three significant changes. First, the version number changed from 2 numbers to 3 (Major, Minor, Patch). Second, the copyright of the original developers was given to Spread Concepts LLC. Third, a number of security bugfixes in the client-daemon code were made.

1. The new spread daemon will work with both old and new client libraries, but clients compiled against the new libraries (3.15.1) will NOT work with old daemons.
2. The SP_version function has had it's interface changed. This breaks source compatibility, but it is a non-essential function and we had to change it sometime. The fix is very simple.

• Make sess_read() non-blocking so slow clients sending messages don't slow or block daemon.
• Implement input validation on all messages received from clients. This is to address the possibility of clients overflowing the servers buffers. These problems were uncovered in a security audit by Olaf Kirch of Caldera.
• Use chroot to isolate spread and drop priveledges if spread is run by root. If it is run as a normal user, then no chroot is done and it runs just as before. The user to run as and the location of the chroot jail can be configured in the spread_params.h file at compile time.
• Fix a bug that causes a significant memory leak when retransmissions are required. This memory leak exists in versions 3.13, 3.14 and 3.15.0.
• SP_version() no longer returns a float, instead it returns in 3 separate parameters the 3 components of the version number.

• New version scheme. We now have three parts to identify the version.
• A bugfix to prevent users from sending a message to too many groups at once.

Version 3.14

• Bugfixes to skiplist implementation.
• Critical fixes to group membership that removes bugs introduced in optimization work of 3.13.
• Critical bugfix to membership which solves problem introduced in 3.13 with using configurations of more then one segment.
• Changes to spread.conf parser, but nothing that should be user visable.
• Better error/problem reporting in a few cases.
• Enforce message length limit. SP_mcast* will now return an error if a message body is too long.
• Minor memory leak at initialization fixed.
• Fix to events.c to prevent an illegal message bug. Bug is triggered when messages are sent very fast during a membership merge.

Version 3.13

• Major scalability improvements. Should now support upto 990 connections to each daemon, thousands of groups, upto 128 daemons per configuration, and double the throughput for small messages.
• A new, more flexible and readable configuration file format. The configuration file is now called 'spread.conf' and has a simplified format that also allows runtime configuration of the log format, which messages to log, and a flag to disable certain dangerous monitor commands.

All known bugs are also fixed with this version. This includes a bug with the new NO_DROP semantics in version 3.12 and some membership crashes that occur under high load. Behind the scenes, this version consolidates some of the research done in the Center for Networking and Distributed Systems lab (http://www.cnds.jhu.edu) at Johns Hopkins University over the last few years into the public distribution.

The Non-commercial license for Spread has been updated to version 1.2. The changes include a clarification about Spread versions, a small addition to the attribution requirement and a capitalization of the word SPREAD.

Currently two freely-available commercial use licenses are available for Spread. One allows the use of Spread for distributed logging of Apache web server logs, and the other allows the use of Spread for distributed SSL session caching with Apache-SSL. For more details please see http://www.spread.org/license/

Version 3.12

• Integrated FreeBSD port. A Big thanks to Ben Laurie ben@algroup.co.uk for this port and a lot of compile warnings, type cleanups and 2 bugfixes.
• Removed some compile warnings.
• 2 bugs fixed. One in SP_group_ids_equal() and one in the daemon that would never trigger in reality.
• Updated java interface to most recent version.
• Updated events (E_* functions) to most recent version. This adds the ability to attach to file descriptors for Exceptions and Write events as well as reads. It also adds a void* data pointer to the callback parameters.
• Change SP_recv semantics to be "No Drop" by default. The old "drop" semantics are still available by setting the service_type parameter to SP_recv to DROP_RECV before calling it. *** See the file "Short_Buffer_Handling.txt" for detailed comments on this change and what application modifications might be necessary ***
• Documentation updates.
• Changed the way arch.h is implemented. This might require some additions for Ports that we do not support natively.
• Added Is_self_leave() macro which matches documentation for testing self-leaves.
• Changed SP_* and E_* functions to use 'const' appropriately.
• Reserved fields are listed in sp.h to prevent collisions.
• Make MAX_PROC_NAME and MAX_PRIVATE_GROUP sizes available in sp.h.
• Added Perl library interface. You can now write Spread client programs in perl. Thanks to Theo Schlossnagle .
• License updated to version 1.1. Main change is a reformatting and updated email addresses.

Version 3.11

Version 3.10

• Expiration Date Removed from Binary version.
  This release of the executables will not expire.
• New Copyright Statement
  The copyright of Spread has been finalized. The statement at the top of this file shows it.
• Bugs Fixed:
  • Bug fix to the SP_scatter_recv() call to make receives into scatter structures of multiple entries of different sizes work.
  • The Linux platform also had scatter-gather functionality enabled (now that glibc has working sendmsg, recvmsg calls)
  • Fixed bug where packets which are delayed a long time in the network but then actually arrive at a spread daemon cause a crash.

Version 3.9

• Multi-threading:
  Libsp.a has split into two libraries, libsp.a and libtsp.a, they are identical except that libtsp.a is thread-safe and libsp.a is not. Both versions are provided so that single-threaded programs who don't need a safe library can avoid the (admittedly pretty small) overhead of locking. The libtsp.a can be used identically with libsp.a with the addition that all SP_* functions can be called by multiple threads at the same time. Thread safety in libtsp.a is provided optimally, with the most possible parallelism allowed. Threads acting on separate mailboxes will never block each other, threads writing to a mailbox will never block other threads reading from the same mailbox and vice versa. The one time thread calls to SP_ functions will be serialzed is when multiple threads are reading the same mailbox or writing the same mailbox. There is minimal locking of internal structures which can temporarily block another thread, but since a thread never blocks while in one of these structure critical sections and the duration of the lock is at most a few instructions the delay should be minimal.
• IP-Multicast:
  Spread now allows you to specifiy IP-Multicast addresses(i.e. 224.0.0.1) as the broadcast address for a segment in the config.dat file. This will work transparently in the same segment, and is guaranteed not to spill multicast packets beyond that segment. If multi-segment ip-multicast is desired, please contact us.
• Fixed several minor bugs.
  
• Included man pages for SP functions. The old html documentation is also included.
• None of these changes effect the Java version which has not changed.

Questions or comments to: webmaster@spread.org TEL: (410) 516-5562 FAX: (410) 516-6134