Add the LZWEncode filter that actually compresses using the LZW method,

rather than the dummy encoder in slzwce.c that expanded the input, but
that avoided the patent issues. Bug 687554.

Also fix a long standing issue in filter closing that failed to flush the
target stream after the final s_process_write_buf with 'last' == true.
Bug 224134.

DETAILS:

The LZWEncode filter was moved over from an older version of Ghostscript
that actually had a working LZWEncode.

The s_filter_close logic (discussed at length in bug 224134) was fixed
to be consistent with the specification and with Adobe. While the
target stream is not closed (per the specification unless CloseTarget is
true), the specification *DOES* call for an automatic 'flushfile' which
is supposed to be propagated down the filter chain.

git-svn-id: http://svn.ghostscript.com/ghostscript/trunk@5379 a1074d23-0009-0410-80fe-cf8c14f379e6

19 files changed, 32 insertions, 210 deletions

diff --git a/gs/doc/Develop.htm b/gs/doc/Develop.htm
index a8ec78d67..45e4f2638 100644
--- a/gs/doc/Develop.htm
+++ b/gs/doc/Develop.htm
@@ -500,8 +500,8 @@ JPX (JPEG 2000):
 Other compression/decompression:
 <dd>
 <a href="../src/slzwc.c">src/slzwc.c</a>,
-<a href="../src/slzwce.c">src/slzwce.c</a>,
 <a href="../src/slzwd.c">src/slzwd.c</a>,
+<a href="../src/slzwe.c">src/slzwe.c</a>,
 <a href="../src/slzwx.h">src/slzwx.h</a>,
 <a href="../src/srld.c">src/srld.c</a>,
 <a href="../src/srle.c">src/srle.c</a>,
diff --git a/gs/doc/Issues.htm b/gs/doc/Issues.htm
index f7528ba9d..f679d7134 100644
--- a/gs/doc/Issues.htm
+++ b/gs/doc/Issues.htm
@@ -105,21 +105,6 @@ class="offsite">#615165</a>, September 26, 2002.
 <hr>
 <h2><a name="Minor_bugs"></a>Minor Bugs.</h2>
 
-<h3> Multiple encode filters in a pipeline fail.</h3>
-The following code fragment should output "3E>". It outputs nothing. 
-<blockquote><pre>
-%!PS 
-(%stdout) (w) file 
-/ASCIIHexEncode filter 
-/ASCIIHexEncode filter 
-closefile 
-</pre></blockquote>
-Bug <a href="http://bugs.ghostscript.com/show_bug.cgi?id=224134"
-class="offsite">#224134</a>, December 2, 2000.
-<p>
-<i>LPD reports that this behavior is in fact according to spec and 
-programs cannot rely on this behavior.</i>
-
 <h3> Bad JPEG stream in PDF generated when source ends prematurely</h3>
 When GS converts the source image to JPEG stream in PDF file and the
 source data end prematurely, it generates bad JPEG stream. 
diff --git a/gs/doc/Projects.htm b/gs/doc/Projects.htm
index 7be31212e..e40bbb334 100644
--- a/gs/doc/Projects.htm
+++ b/gs/doc/Projects.htm
@@ -210,8 +210,7 @@ entirely redone later): compressing images.  Currently the driver only
 compresses character bitmaps, and doesn't compress other images at all.
 <I>It should use the <b><tt>CCITTFaxEncode</tt></b> filter for 1-bit-deep
 images, and plane-separated <b><tt>LZWEncode</tt></b> compression for color
-images, using the <b><tt>miGIF</tt></b> algorithms that are believed to be
-free of the Welch patent for the latter.  When generating LL3 PS, the
+images.  When generating LL3 PS, the
 <b><tt>Flate</tt></b> compression will work better than miGIF. It may be
 worth trying several methods on each image and use the one that works best.</I>
 
diff --git a/gs/doc/Ps2pdf.htm b/gs/doc/Ps2pdf.htm
index 318c80416..f5020f00d 100644
--- a/gs/doc/Ps2pdf.htm
+++ b/gs/doc/Ps2pdf.htm
@@ -397,10 +397,11 @@ Flate compression.  If you want to force JPEG compression, use
 <p>
 <a name="note_2">(note 2)</a>
 Because of Unisys's threats regarding the Welch patent,
-<b><tt>ps2pdf</tt></b> cannot actually use LZW compression: instead, it
+<b><tt>ps2pdf</tt></b> does not actually use LZW compression: instead, it
 treats all requests for LZW compression as calling for Flate compression.
 Concomitantly, <tt>UseFlateCompression</tt> is treated as always on, and
-the value of this parameter is ignored as with note 0.
+the value of this parameter is ignored as with note 0. Now that the patent
+has expired, we could change this should it become worthwhile.
 
 <p>
 <a name="note_3">(note 3)</a>
diff --git a/gs/src/bcwin32.mak b/gs/src/bcwin32.mak
index e718781be..ba3378315 100644
--- a/gs/src/bcwin32.mak
+++ b/gs/src/bcwin32.mak
@@ -325,8 +325,7 @@ COMPILE_INITS=0
 BAND_LIST_STORAGE=file
 
 # Choose which compression method to use when storing band lists in memory.
-# The choices are 'lzw' or 'zlib'.  lzw is not recommended, because the
-# LZW-compatible code in Ghostscript doesn't actually compress its input.
+# The choices are 'lzw' or 'zlib'.
 
 BAND_LIST_COMPRESSOR=zlib
 
diff --git a/gs/src/dvx-gcc.mak b/gs/src/dvx-gcc.mak
index 411b06538..80a95c5af 100644
--- a/gs/src/dvx-gcc.mak
+++ b/gs/src/dvx-gcc.mak
@@ -277,8 +277,7 @@ COMPILE_INITS=0
 BAND_LIST_STORAGE=file
 
 # Choose which compression method to use when storing band lists in memory.
-# The choices are 'lzw' or 'zlib'.  lzw is not recommended, because the
-# LZW-compatible code in Ghostscript doesn't actually compress its input.
+# The choices are 'lzw' or 'zlib'. 
 
 BAND_LIST_COMPRESSOR=zlib
 
diff --git a/gs/src/lib.mak b/gs/src/lib.mak
index 4d88805c9..3d1379bd8 100644
--- a/gs/src/lib.mak
+++ b/gs/src/lib.mak
@@ -1319,7 +1319,7 @@ $(GLOBJ)sddparam.$(OBJ) : $(GLSRC)sddparam.c $(AK) $(std_h)\
 # ---------------- LZW filters ---------------- #
 # These are used by Level 2 in general.
 
-lzwe_=$(GLOBJ)slzwce.$(OBJ) $(GLOBJ)slzwc.$(OBJ)
+lzwe_=$(GLOBJ)slzwe.$(OBJ) $(GLOBJ)slzwc.$(OBJ)
 $(GLD)lzwe.dev : $(LIB_MAK) $(ECHOGS_XE) $(lzwe_)
 	$(SETMOD) $(GLD)lzwe $(lzwe_)
 
@@ -1327,9 +1327,9 @@ $(GLD)lzwe.dev : $(LIB_MAK) $(ECHOGS_XE) $(lzwe_)
 $(GLD)slzwe.dev : $(GLD)lzwe.dev
 	$(CP_) $(GLD)lzwe.dev $(GLD)slzwe.dev
 
-$(GLOBJ)slzwce.$(OBJ) : $(GLSRC)slzwce.c $(AK) $(stdio__h) $(gdebug_h)\
+$(GLOBJ)slzwe.$(OBJ) : $(GLSRC)slzwe.c $(AK) $(stdio__h) $(gdebug_h)\
  $(slzwx_h) $(strimpl_h)
-	$(GLCC) $(GLO_)slzwce.$(OBJ) $(C_) $(GLSRC)slzwce.c
+	$(GLCC) $(GLO_)slzwe.$(OBJ) $(C_) $(GLSRC)slzwe.c
 
 $(GLOBJ)slzwc.$(OBJ) : $(GLSRC)slzwc.c $(AK) $(std_h)\
  $(slzwx_h) $(strimpl_h)
diff --git a/gs/src/macos-mcp.mak b/gs/src/macos-mcp.mak
index 5c2df64f2..11f98cc51 100644
--- a/gs/src/macos-mcp.mak
+++ b/gs/src/macos-mcp.mak
@@ -190,8 +190,7 @@ COMPILE_INITS=0
 BAND_LIST_STORAGE=file
 
 # Choose which compression method to use when storing band lists in memory.
-# The choices are 'lzw' or 'zlib'.  lzw is not recommended, because the
-# LZW-compatible code in Ghostscript doesn't actually compress its input.
+# The choices are 'lzw' or 'zlib'.  
 
 BAND_LIST_COMPRESSOR=zlib
 
diff --git a/gs/src/macosx.mak b/gs/src/macosx.mak
index caec19522..be11d0846 100644
--- a/gs/src/macosx.mak
+++ b/gs/src/macosx.mak
@@ -343,8 +343,7 @@ COMPILE_INITS=0
 BAND_LIST_STORAGE=file
 
 # Choose which compression method to use when storing band lists in memory.
-# The choices are 'lzw' or 'zlib'.  lzw is not recommended, because the
-# LZW-compatible code in Ghostscript doesn't actually compress its input.
+# The choices are 'lzw' or 'zlib'.
 
 BAND_LIST_COMPRESSOR=zlib
 
diff --git a/gs/src/msvc32.mak b/gs/src/msvc32.mak
index f7655e5d5..0e3f8caa6 100644
--- a/gs/src/msvc32.mak
+++ b/gs/src/msvc32.mak
@@ -576,8 +576,7 @@ BAND_LIST_STORAGE=file
 !endif
 
 # Choose which compression method to use when storing band lists in memory.
-# The choices are 'lzw' or 'zlib'.  lzw is not recommended, because the
-# LZW-compatible code in Ghostscript doesn't actually compress its input.
+# The choices are 'lzw' or 'zlib'.
 
 !ifndef BAND_LIST_COMPRESSOR
 BAND_LIST_COMPRESSOR=zlib
diff --git a/gs/src/msvclib.mak b/gs/src/msvclib.mak
index 9106b042b..434438440 100644
--- a/gs/src/msvclib.mak
+++ b/gs/src/msvclib.mak
@@ -389,8 +389,7 @@ BAND_LIST_STORAGE=file
 !endif
 
 # Choose which compression method to use when storing band lists in memory.
-# The choices are 'lzw' or 'zlib'.  lzw is not recommended, because the
-# LZW-compatible code in Ghostscript doesn't actually compress its input.
+# The choices are 'lzw' or 'zlib'.
 
 !ifndef BAND_LIST_COMPRESSOR
 BAND_LIST_COMPRESSOR=zlib
diff --git a/gs/src/openvms.mak b/gs/src/openvms.mak
index a2ca85daa..7c791e94e 100644
--- a/gs/src/openvms.mak
+++ b/gs/src/openvms.mak
@@ -257,8 +257,7 @@ COMPILE_INITS=0
 BAND_LIST_STORAGE=file
 
 # Choose which compression method to use when storing band lists in memory.
-# The choices are 'lzw' or 'zlib'.  lzw is not recommended, because the
-# LZW-compatible code in Ghostscript doesn't actually compress its input.
+# The choices are 'lzw' or 'zlib'.
 
 BAND_LIST_COMPRESSOR=zlib
 
diff --git a/gs/src/os2.mak b/gs/src/os2.mak
index 5688f9804..9817d2fb4 100644
--- a/gs/src/os2.mak
+++ b/gs/src/os2.mak
@@ -454,8 +454,7 @@ COMPILE_INITS=0
 BAND_LIST_STORAGE=file
 
 # Choose which compression method to use when storing band lists in memory.
-# The choices are 'lzw' or 'zlib'.  lzw is not recommended, because the
-# LZW-compatible code in Ghostscript doesn't actually compress its input.
+# The choices are 'lzw' or 'zlib'.
 
 BAND_LIST_COMPRESSOR=zlib
 
diff --git a/gs/src/slzwce.c b/gs/src/slzwce.c
deleted file mode 100644
index 8fbdc2bb3..000000000
--- a/gs/src/slzwce.c
+++ /dev/null
@@ -1,159 +0,0 @@
-/* Copyright (C) 1994, 1995, 1996, 1998, 1999 Aladdin Enterprises.  All rights reserved.
-  
-  This software is provided AS-IS with no warranty, either express or
-  implied.
-  
-  This software is distributed under license and may not be copied,
-  modified or distributed except as expressly authorized under the terms
-  of the license contained in the file LICENSE in this distribution.
-  
-  For more information about licensing, please refer to
-  http://www.ghostscript.com/licensing/. For information on
-  commercial licensing, go to http://www.artifex.com/licensing/ or
-  contact Artifex Software, Inc., 101 Lucas Valley Road #110,
-  San Rafael, CA  94903, U.S.A., +1(415)492-9861.
-*/
-
-/* $Id$ */
-/* Simple encoder compatible with LZW decoding filter */
-#include "stdio_.h"		/* includes std.h */
-#include "gdebug.h"
-#include "strimpl.h"
-#include "slzwx.h"
-
-/* ------ Alternate LZWEncode filter implementation ------ */
-
-/*
-
-   The encoded data stream produced by this implementation of the LZWEncode
-   filter consists of a sequence of 9-bit data elements.  These elements are
-   packed into bytes in big-endian order, e.g. the elements
-
-   100000000 001100001
-
-   occurring at the very beginning of the data stream would be packed into
-   bytes as
-
-   10000000 00011000 01......
-
-   The first bit of each data element is a control bit.  If the control bit is
-   0, the remaining 8 bits of the data element are a data byte.  If the control
-   bit is 1, the remaining 8 bits of the data element define a control
-   function:
-
-   1 00000000   synchronization mark, see below
-   1 00000001   end of data
-   1 xxxxxxxx   not used (all other values)
-
-   The synchronization mark occurs at the beginning of the data stream, and at
-   least once every 254 data bytes thereafter.
-
-   This format is derived from basic principles of data encoding (the use of a
-   separate flag bit to distinguish out-of-band control information from data
-   per se, and the use of a periodic synchronization mark to help verify the
-   validity of a data stream); it has no relationship to data compression.  It
-   is, however, compatible with LZW decompressors.  It produces output that is
-   approximately 9/8 times the size of the input.
-
- */
-
-/* Define the special codes, relative to 1 << InitialCodeLength. */
-#define CODE_RESET 0
-#define CODE_EOD 1
-#define CODE_0 2		/* first assignable code */
-
-/* Internal routine to put a code into the output buffer. */
-/* Let S = ss->code_size. */
-/* Relevant invariants: 9 <= S <= 15, 0 <= code < 1 << S; */
-/* 1 <= ss->bits_left <= 8; only the rightmost (8 - ss->bits_left) */
-/* bits of ss->bits contain valid data. */
-private byte *
-lzw_put_code(register stream_LZW_state * ss, byte * q, uint code)
-{
-    uint size = ss->code_size;
-    byte cb = (ss->bits << ss->bits_left) +
-	(code >> (size - ss->bits_left));
-
-    if_debug2('W', "[w]writing 0x%x,%d\n", code, ss->code_size);
-    *++q = cb;
-    if ((ss->bits_left += 8 - size) <= 0) {
-	*++q = code >> -ss->bits_left;
-	ss->bits_left += 8;
-    }
-    ss->bits = code;
-    return q;
-}
-
-/* Initialize LZW-compatible encoding filter. */
-private int
-s_LZWE_reset(stream_state * st)
-{
-    stream_LZW_state *const ss = (stream_LZW_state *) st;
-
-    ss->code_size = ss->InitialCodeLength + 1;
-    ss->bits_left = 8;
-    /* Force the first code emitted to be a reset. */
-    ss->next_code = (1 << ss->code_size) - 2;
-    return 0;
-}
-private int
-s_LZWE_init(stream_state * st)
-{
-    stream_LZW_state *const ss = (stream_LZW_state *) st;
-
-    ss->InitialCodeLength = 8;
-    ss->table.encode = 0;
-    return s_LZWE_reset(st);
-}
-
-/* Process a buffer */
-private int
-s_LZWE_process(stream_state * st, stream_cursor_read * pr,
-	       stream_cursor_write * pw, bool last)
-{
-    stream_LZW_state *const ss = (stream_LZW_state *) st;
-    register const byte *p = pr->ptr;
-    const byte *rlimit = pr->limit;
-    register byte *q = pw->ptr;
-    byte *wlimit = pw->limit;
-    int status = 0;
-    int signal = 1 << (ss->code_size - 1);
-    uint limit_code = (1 << ss->code_size) - 2;		/* reset 1 early */
-    uint next_code = ss->next_code;
-
-    while (p < rlimit) {
-	if (next_code == limit_code) {	/* Emit a reset code. */
-	    if (wlimit - q < 2) {
-		status = 1;
-		break;
-	    }
-	    q = lzw_put_code(ss, q, signal + CODE_RESET);
-	    next_code = signal + CODE_0;
-	}
-	if (wlimit - q < 2) {
-	    status = 1;
-	    break;
-	}
-	q = lzw_put_code(ss, q, *++p);
-	next_code++;
-    }
-    if (last && status == 0) {
-	if (wlimit - q < 2)
-	    status = 1;
-	else {
-	    q = lzw_put_code(ss, q, signal + CODE_EOD);
-	    if (ss->bits_left < 8)
-		*++q = ss->bits << ss->bits_left;	/* final byte */
-	}
-    }
-    ss->next_code = next_code;
-    pr->ptr = p;
-    pw->ptr = q;
-    return status;
-}
-
-/* Stream template */
-const stream_template s_LZWE_template = {
-    &st_LZW_state, s_LZWE_init, s_LZWE_process, 1, 2, NULL,
-    s_LZW_set_defaults, s_LZWE_reset
-};
diff --git a/gs/src/stream.c b/gs/src/stream.c
index 95ed5d767..1a12d4d55 100644
--- a/gs/src/stream.c
+++ b/gs/src/stream.c
@@ -325,16 +325,28 @@ s_filter_write_flush(register stream * s)
 }
 
 /* Close a filter.  If this is an encoding filter, flush it first. */
+/* If CloseTarget was specified (close_strm), then propagate the sclose */
 int
 s_filter_close(register stream * s)
 {
+    int status, close=s->close_strm;
+    stream *stemp = s->strm;
+
     if (s_is_writing(s)) {
 	int status = s_process_write_buf(s, true);
 
 	if (status != 0 && status != EOFC)
 	    return status;
+        status = sflush(stemp);
+	if (status != 0 && status != EOFC)
+	    return status;
     }
-    return s_std_close(s);
+    status = s_std_close(s);
+    if (status != 0 && status != EOFC)
+	return status;
+    if (close && stemp != 0)
+	return sclose(stemp);
+    return status;
 }
 
 /* Disregard a stream error message. */
diff --git a/gs/src/unix-gcc.mak b/gs/src/unix-gcc.mak
index f8cf9abd0..4ae1693d7 100644
--- a/gs/src/unix-gcc.mak
+++ b/gs/src/unix-gcc.mak
@@ -340,8 +340,7 @@ COMPILE_INITS=0
 BAND_LIST_STORAGE=file
 
 # Choose which compression method to use when storing band lists in memory.
-# The choices are 'lzw' or 'zlib'.  lzw is not recommended, because the
-# LZW-compatible code in Ghostscript doesn't actually compress its input.
+# The choices are 'lzw' or 'zlib'.
 
 BAND_LIST_COMPRESSOR=zlib
 
diff --git a/gs/src/unixansi.mak b/gs/src/unixansi.mak
index 80901df34..89cffa35e 100644
--- a/gs/src/unixansi.mak
+++ b/gs/src/unixansi.mak
@@ -316,8 +316,7 @@ COMPILE_INITS=0
 BAND_LIST_STORAGE=file
 
 # Choose which compression method to use when storing band lists in memory.
-# The choices are 'lzw' or 'zlib'.  lzw is not recommended, because the
-# LZW-compatible code in Ghostscript doesn't actually compress its input.
+# The choices are 'lzw' or 'zlib'.
 
 BAND_LIST_COMPRESSOR=zlib
 
diff --git a/gs/src/watcw32.mak b/gs/src/watcw32.mak
index 193fd0fe7..59dcc0bb9 100644
--- a/gs/src/watcw32.mak
+++ b/gs/src/watcw32.mak
@@ -226,8 +226,7 @@ COMPILE_INITS=0
 BAND_LIST_STORAGE=file
 
 # Choose which compression method to use when storing band lists in memory.
-# The choices are 'lzw' or 'zlib'.  lzw is not recommended, because the
-# LZW-compatible code in Ghostscript doesn't actually compress its input.
+# The choices are 'lzw' or 'zlib'.
 
 BAND_LIST_COMPRESSOR=zlib
 
diff --git a/gs/src/zfilter2.c b/gs/src/zfilter2.c
index 9eaba2fbe..7fc87325a 100644
--- a/gs/src/zfilter2.c
+++ b/gs/src/zfilter2.c
@@ -130,11 +130,6 @@ filter_write_predictor(i_ctx_t *i_ctx_p, int npop,
 
 /* <target> LZWEncode/filter <file> */
 /* <target> <dict> LZWEncode/filter <file> */
-/*
- * Note: the default implementation of this filter, in slzwce.c,
- * does not use any algorithms that could reasonably be claimed
- * to be subject to Unisys' Welch Patent.
- */
 private int
 zLZWE(i_ctx_t *i_ctx_p)
 {