-
Notifications
You must be signed in to change notification settings - Fork 4
/
Copy pathChangeLog
2546 lines (2392 loc) · 129 KB
/
ChangeLog
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
--------------------------------------------------------------------
ChangeLog for Privoxy
--------------------------------------------------------------------
*** Version 3.0.28 stable ***
- Bug fixes for regressions in 3.0.27:
- Fixed misplaced parentheses.
Reported by David Binderman.
- Changed two regression tests to depend on config directive
enable-remote-toggle instead of FEATURE_TOGGLE.
*** Version 3.0.27 stable ***
- General improvements:
- Add a receive-buffer-size directive which can be used to
set the size of the previously statically allocated buffer
in handle_established_connection().
Increasing the buffer size increases Privoxy's memory usage but
can lower the number of context switches and thereby reduce the
CPU usage and potentially increase the throughput.
This is mostly relevant for fast network connections and
large downloads that don't require filtering.
Sponsored by: Robert Klemme
- Add a listen-backlog directive which specifies the backlog
value passed to listen().
Sponsored by: Robert Klemme
- Add an enable-accept-filter directive which allows to
toggle accept filter support at run time when compiled
with FEATURE_ACCEPT_FILTER support.
It makes testing more convenient and now that it's
optional we can emit an error message if enabling
the accept filter fails.
Sponsored by: Robert Klemme
- Add a delay-response{} action.
This is useful to tar pit JavaScript requests that
are endlessly retried in case of blocks. It can also
be used to simulate a slow Internet connection.
Sponsored by: Robert Klemme
- Add a 'trusted-cgi-referrer' directive.
It allows to configure another page or site that can be used
to reach sensitive CGI resources.
Sponsored by: Robert Klemme
- Add a --fuzz mode which exposes Privoxy internals to input
from files or stdout.
Mainly tested with American Fuzzy Lop. For details see:
https://www.fabiankeil.de/talks/fuzzing-on-freebsd/
This work was partially funded with donations and done
as part of the Privoxy month in 2015.
- Consistently use the U(ngreedy) flag in the 'img-reorder' filter.
- listen_loop(): Reuse a single thread attribute object
The object doesn't change and creating a new one for
every thread is a waste of (CPU) time.
Sponsored by: Robert Klemme
- Free csp resources in the thread that belongs to the csp instead
of the main thread which has enough on its plate already.
Sponsored by: Robert Klemme
- Improve 'socket timeout reached' message.
Log the timeout that was triggered and downgrade the
log level to LOG_LEVEL_CONNECT to reduce the log noise
with common debug settings.
The timeout isn't necessary the result of an error and
usually merely indicates that Privoxy's socket timeout
is lower than the relevant timeouts used by client and
server.
Sponsored by: Robert Klemme
- Explicitly taint the server socket in case of CONNECT requests.
This doesn't fix any known problems, but makes
some log messages less confusing.
- Let write_pid_file() terminate if the pid file can't be opened.
Logging the issue at info level is unlikely to help.
- log_error(): Reduce the mutex-protected area by not using a
heap-allocated buffer that is shared between all threads.
This increases performance and reduces the latency with
verbose debug settings and multiple concurrent connections.
Sponsored by: Robert Klemme
- Let zalloc() use calloc() if it's available.
In some situations using calloc() can be faster than
malloc() + memset() and it should never be slower.
In the real world the impact of this change is not
expected to be noticeable.
Sponsored by: Robert Klemme
- Never use select() when poll() is available.
On most platforms select() is limited by FD_SETSIZE while
poll() is not. This was a scaling issue for multi-user setups.
Using poll() has no downside other than the usual risk
that code modifications may introduce new bugs that have
yet to be found and fixed.
At least in theory this commit could also reduce the latency
when there are lots of connections and select() would use
"bit fields in arrays of integers" to store file descriptors.
Another side effect is that Privoxy no longer has to stop
monitoring the client sockets when pipelined requests are
waiting but can't be read yet.
This code keeps the select()-based code behind ifdefs for
now but hopefully it can be removed soonish to make the
code more readable.
Sponsored by: Robert Klemme
- Add a 'reproducible-tarball-dist' target.
It's currently separate from the "tarball-dist" target
because it requires a tar implementation with mtree spec
support.
It's far from being perfect and does not enforce a
reproducible mode, but it's better than nothing.
- Use arc4random() if it's available.
While Privoxy doesn't need high quality pseudo-random numbers
there's no reason not to use them when we can and this silences
a warning emitted by code checkers that can't tell whether or not
the quality matters.
- Show the FEATURE_EXTERNAL_FILTERS status on the status page.
Better late than never. Previously a couple of tests weren't
executed as Privoxy-Regression-Test couldn't detect that the
FEATURE_EXTERNAL_FILTERS dependency was satisfied.
- Ditch FEATURE_IMAGE_DETECT_MSIE.
It's an obsolete workaround we inherited from Junkbuster
and was already disabled by default.
Users that feel the urge to work around issues with
image requests coming from an Internet Explorer version
from more than 15 years ago can still do this using tags.
- Consistently use strdup_or_die() instead of strdup() in
cases where allocation failures aren't expected.
Using strdup_or_die() allows to remove a couple of explicit
error checks which slightly reduces the size of the binary.
- Insert a refresh tag into the /client-tags CGI page when
serving it while a client-specific tag is temporarily enabled.
This makes it less likely that the user ends up
looking at tag state that is out of date.
- Use absolute URLs in the client-tag forms.
It's more consistent with the rest of the CGI page
URLs and makes it more convenient to copy the forms
to external pages.
- cgi_error_disabled(): Use status code 403 and an appropriate response line
- Use a dedicated CGI handler to deal with tag-toggle requests
As a result the /client-tags page is now safe to reach without
trusted Referer header which makes bookmarking or linking to
it more convenient.
Finally, refreshing the /client-tags page to show the
current state can no longer unintentionally repeat the
previous toggle request.
- Don't add a "Connection" header for CONNECT requests.
Explicitly sending "Connection: close" is not necessary and
apparently it causes problems with some forwarding proxies
that will close the connection prematurely.
Reported by Marc Thomas.
- Fix compiler warnings.
- Bug fixes:
- rfc2553_connect_to(): Properly detect and log when poll()
reached the time out. Previously this was logged as:
Could not connect to [...]: No error: 0.
which isn't very helpful.
Sponsored by: Robert Klemme
- add_tag_for_client(): Set time_to_live properly.
Previously the time_to_live was always set for the first tag.
Attempts to temporarily enable a tag would result in enabling
it permanently unless no tag was enabled already.
- Revert r1.165 which didn't perform as advertised.
While the idea was to use "https:// when creating links
for the user manual on the website", the actual effect
was to use "https://" when Privoxy was supposed to serve
the user manual itself.
Reported by Yossi Zahn on Privoxy-devel@.
- socks5_connect(): Fail in case of unsupported address types.
Previously they would not be detected right away and
Privoxy would fail later on with an error message that
didn't make it obvious that the problem was socks-related.
So far, no such problems have actually been reported.
- socks5_connect(): Properly deal with socks replies that
contain IPv6 addresses.
Previously parts of the reply were left unread and
later on treated as invalid HTTP response data.
Fixes #904 reported by Danny Goossen who also provided
the initial version of this patch.
- Action file improvements:
- Unblock 'msdn.microsoft.com/'.
It (presumably) isn't used to serve the kind of ads Privoxy should
block by default but happens to serve lots of pages with URLs that
are likely to result in false positives.
Reported by bugreporter1694 in AF#939.
- Disable gif deanimation for requests tagged with CSS-REQUEST.
The action will ignore content that isn't considered text
anyway and explicitly disabling it makes this more obvious
if "action" debugging (debug 65536) is enabled while
"gif deanimation" debugging (debug 256) isn't.
- Explicitly disable HTML filters for requests with CSS-REQUEST tag.
The filters are unlikely to break CSS files but executing
them without (intentionally) getting any hits is a waste of
cpu time and makes the log more noisy when running with
"debug 64".
- Unblock 'adventofcode.com/'.
Reported by Clint Adams in Debian bug #848211.
Fixes Roland's AF#937.
- Unblock 'adlibris.com'.
Reported by Wyrex in #935
- Unblock .golang.org/
- Add fast-redirects exception for '.youtube.com/.*origin=http'
- Privoxy-Log-Parser:
- Don't gather host and resource statistics if they aren't requested.
While the performance impact seems negligible this significantly
reduces the memory usage if there are lots of requests.
- Bump version as the behaviour (slightly) changed.
- Count connection failures as well in statistics mode.
Sponsored by: Robert Klemme
- Count connection timeouts as well in statistics mode.
Sponsored by: Robert Klemme
- Fix an 'uninitialized value' warning when generating
statistics for a log file without response headers.
While privoxy-log-parser was supposed to detect this already,
the check was flawed and the message the user didn't see was
somewhat confusing anyway.
Now the message is less confusing, more helpful and actually printed.
Reported by: Robert Klemme
- Documentation improvements:
- Refer to the git sources instead of CVS.
- Use GNU/Linux when referring to the OS instead of the kernel.
- Add FAQ entry for what to do if editing the config file is access denied.
- Add brief HTTP/2 FAQ.
- Add a small fuzzing section to the developer documentation.
- Add a client-header-tagger{client-ip-address} example.
- Stop suggesting that Privoxy is an anonymizing proxy.
The term could lead to Privoxy users overestimating
what it can do on its own (without Tor).
- Make it more obvious that SPI accepts Paypal, too.
Currently most donations are made through the Paypal account
managed by Zwiebelfreunde e.V. and a more even distribution
would be useful.
- Suggest to log applying actions as well when reproducing problems.
- Explicitly mention that Privoxy binaries are built by individuals
on their own systems. Buyer beware!
- Mention the release feed on the homepage.
- Remove a mysterious comment with a GNU FDL link as it isn't
useful and could confuse license scanners.
In May 2002 it was briefly claimed that "this document" was covered
by the GNU FDL. The commit message (r1.5) doesn't explain the motivation
or whether all copyright holders were actually asked and agreed to the
declared license change.
It's thus hard to tell whether or not the license change was legit,
but luckily two days later the "doc license" was "put" "back to GPL"
anyway (r1.6).
At the same time the offending comment with a link to the FDL
(not the GPL) was added for no obvious reason.
Now it's gone again.
- Regression tests:
- Bump for-privoxy-version to 3.0.27 as we now rely on untrusted
CGI request being rejected with status code 403 (instead of 200).
- Update test for /send-stylesheet and add another one
- Templates:
- Consistently use https:// when linking to the Privoxy website.
- Remove SourceForge references in Copyright header.
- Remove a couple of SourceForge references in a comment.
While at it, fix the grammar.
- Move the site-specific documentation block before the generic one.
While most Privoxy installations don't have a site-specific
documentation block, in cases were it exists it's likely to
be more relevant than the generic one.
Showing it first makes it less likely that users stop reading
before they reach it, especially on pages that don't fit on
the screen.
- Build system improvements:
- Prefer openjade to jade. On some systems Jade produces
HTML with unescaped ampersands in URLs.
- Prefer OpenSP to SP to be consistent.
- Have Docbook generated HTML files be straight ASCII.
Dealing with a mixture of ISO-8859 and UTF-8 files is problematic.
- Echo the filename to stderr for 'make dok-tidy'.
Make it a bit easier to find errors in docbook generated HTML.
- Warn when still using select().
- Warn when compiling without calloc().
- Make it more obvious that the --with-fdsetsize configure switch
is pointless if poll() is available.
- Remove support for AmigaOS.
- Update windows build system to use supported software.
The cygwin gcc -mno-cygwin option is no longer supported, so
convert the windows build system to use the cygwin cross-compiler
to build "native" code.
- Add --enable-static-linking option for configure
does the same thing as LDFLAGS=-static; ./configure
but nicer than mixing evars and configure options.
*** Version 3.0.26 stable ***
- Bug fixes:
- Fixed crashes with "listen-addr :8118" (SF Bug #902).
The regression was introduced in 3.0.25 beta and reported
by Marvin Renich in Debian bug #834941.
- General improvements:
- Log when privoxy is toggled on or off via cgi interface.
- Highlight the "Info: Now toggled " on/off log message
in the Windows log viewer.
- Highlight the loading actions/filter file log message
in the Windows log viewer.
- Mention client-specific tags on the toggle page as a
potentionally more appropriate alternative.
- Documentation improvements:
- Update download section on the homepage.
The downloads are available from the website now.
- Add sponsor FAQ.
- Remove obsolete reference to mailing lists hosted at SourceForge.
- Update the "Before the Release" section of the developer manual.
- Infrastructure improvements:
- Add perl script to generate an RSS feed for the packages
Submitted by "Unknown".
- Build system improvements:
- strptime.h: fix a compiler warning about ambiguous else.
- configure.in: Check for Docbook goo on the BSDs as well.
- GNUMakefile.in: Let the dok-user target remove temporary files.
*** Version 3.0.25 beta ***
- Bug fixes:
- Always use the current toggle state for new requests.
Previously new requests on reused connections inherited
the toggle state from the previous request even though
the toggle state could have changed.
Reported by Robert Klemme.
- Fixed two buffer-overflows in the (deprecated) static
pcre code. These bugs are not considered security issues
as the input is trusted.
Found with afl-fuzz and ASAN.
- General improvements:
- Added support for client-specific tags which allow Privoxy
admins to pre-define tags that are set for all requests from
clients that previously opted in through the CGI interface.
They are useful in multi-user setups where admins may
want to allow users to disable certain actions and filters
for themselves without affecting others.
In single-user setups they are useful to allow more fine-grained
toggling. For example to disable request blocking while still
crunching cookies, or to disable experimental filters only.
This is an experimental feature, the syntax and behaviour may
change in future versions.
Sponsored by Robert Klemme.
- Dynamic filters and taggers now support a $listen-address variable
which contains the address the request came in on.
For external filters the variable is called $PRIVOXY_LISTEN_ADDRESS.
Original patch contributed by pursievro.
- Add client-header-tagger 'listen-address'.
- Include the listen-address in the log message when logging new requests.
Patch contributed by pursievro.
- Turn invalid max-client-connections values into fatal errors.
- The show-status page now shows whether or not dates before 1970
and after 2038 are expected to be handled properly.
This is mainly useful for Privoxy-Regression-Test but could
also come handy when dealing with time-related support requests.
- On Mac OS X the thread id in log messages are more likely to
be unique now.
- When complaining about missing filters, the filter type is logged
as well.
- A couple of harmless coverity warnings were silenced
(CID #161202, CID #161203, CID #161211).
- Action file improvements:
- Filtering is disabled for Range requests to let download resumption
and Windows updates work with the default configuration.
- Unblock ".ardmediathek.de/".
Reported by ThTomate in #932.
- Documentation improvements:
- Add FAQ entry for crashes caused by memory limits.
- Remove obsolete FAQ entry about a bug in PHP 4.2.3.
- Mention the new mailing lists were appropriate.
As the archives have not been migrated, continue to
mention the archives at SF in the contacting section
for now.
- Note that the templates should be adjusted if Privoxy is
running as intercepting proxy without getting all requests.
- A bunch of links were converted to https://.
- Rephrase onion service paragraph to make it more obvious
that Tor is involved and that the whole website (and not
just the homepage) is available as onion service.
- Streamline the "More information" section on the homepage further
by additionally ditching the link to the 'See also' section
of the user manual. The section contains mostly links that are
directly reachable from the homepage already and the rest is
not significant enough to get a link from the homepage.
- Change the add-header{} example to set the DNT header
and use a complete section to make copy and pasting
more convenient.
Add a comment to make it obvious that adding the
header is not recommended for obvious reasons.
Using the DNT header as example was suggested by
Leo Wzukw.
- Streamline the support-and-service template
Instead of linking to the various support trackers
(whose URLs hopefully change soon), link to the
contact section of the user manual to increase the
chances that users actually read it.
- Add a FAQ entry for tainted sockets.
- More sections in the documentation have stable URLs now.
- FAQ: Explain why 'ping config.privoxy.org' is not expected
to reach a local Privoxy installation.
- Note that donations done through Zwiebelfreunde e.V. currently
can't be checked automatically.
- Updated section regarding starting Privoxy under OS X.
- Use dedicated start instructions for FreeBSD and ElectroBSD.
- Removed release instructions for AIX. They haven't been working
for years and unsurprisingly nobody seems to care.
- Removed obsolete reference to the solaris-dist target.
- Updated the release instructions for FreeBSD.
- Removed unfinished release instructions for Amiga OS and HP-UX 11.
- Added a pointer to the Cygwin Time Machine for getting the last release of
Cygwin version 1.5 to use for building Privoxy on Windows.
- Various typos have been fixed.
- Infrastructure improvements:
- The website is no longer hosted at SourceForge and
can be reached through https now.
- The mailing lists at SourceForge have been deprecated,
you can subscribe to the new ones at: https://lists.privoxy.org/
- Migrating the remaining services from SourceForge is
work in progress (TODO list item #53).
- Build system improvements:
- Add configure argument to optimistically redefine FD_SETSIZE
with the intent to change the maximum number of client
connections Privoxy can handle. Only works with some libcs.
Sponsored by Robert Klemme.
- Let the tarball-dist target skip files in ".git".
- Let the tarball-dist target work in cwds other than current.
- Make the 'clean' target faster when run from a git repository.
- Include tools in the generic distribution.
- Let the gen-dist target work in cwds other than current.
- Sort find output that is used for distribution tarballs
to get reproducible results.
- Don't add '-src' to the name of the tar ball generated by the
gen-dist target. The package isn't a source distribution but a
binary package.
While at it, use a variable for the name to reduce the chances
that the various references get out of sync and fix the gen-upload
target which was looking in the wrong directory.
- Add regression-tests.action to the files that are distributed.
- The gen-dist target which was broken since 2002 (r1.92) has been fixed.
- Remove genclspec.sh which has been obsolete since 2009.
- Remove obsolete reference to Redhat spec file.
- Remove the obsolete announce target which has been commented out years ago.
- Let rsync skip files if the checksums match.
- Privoxy-Regression-Test:
- Add a "Default level offset" directive which can be used to
change the default level by a given value.
This directive affects all tests located after it until the end
of the file or a another "Default level offset" directive is reached.
The purpose of this directive is to make it more convenient to skip
similar tests in a given file without having to remove or disable
the tests completely.
- Let test level 17 depend on FEATURE_64_BIT_TIME_T
instead of FEATURE_PTHREAD which has no direct connection
to the time_t size.
- Fix indentation in perldoc examples.
- Don't overlook directives in the first line of the action file.
- Bump version to 0.7.
- Fix detection of the Privoxy version now that https://
is used for the website.
*** Version 3.0.24 stable ***
- Security fixes (denial of service):
- Prevent invalid reads in case of corrupt chunk-encoded content.
CVE-2016-1982. Bug discovered with afl-fuzz and AddressSanitizer.
- Remove empty Host headers in client requests.
Previously they would result in invalid reads. CVE-2016-1983.
Bug discovered with afl-fuzz and AddressSanitizer.
- Bug fixes:
- When using socks5t, send the request body optimistically as well.
Previously the request body wasn't guaranteed to be sent at all
and the error message incorrectly blamed the server.
Fixes #1686 reported by Peter Müller and G4JC.
- Fixed buffer scaling in execute_external_filter() that could lead
to crashes. Submitted by Yang Xia in #892.
- Fixed crashes when executing external filters on platforms like
Mac OS X. Reported by Jonathan McKenzie on ijbswa-users@.
- Properly parse ACL directives with ports when compiled with HAVE_RFC2553.
Previously the port wasn't removed from the host and in case of
'permit-access 127.0.0.1 example.org:80' Privoxy would try (and fail)
to resolve "example.org:80" instead of example.org.
Reported by Pak Chan on ijbswa-users@.
- Check requests more carefully before serving them forcefully
when blocks aren't enforced. Privoxy always adds the force token
at the beginning of the path, but would previously accept it anywhere
in the request line. This could result in requests being served that
should be blocked. For example in case of pages that were loaded with
force and contained JavaScript to create additionally requests that
embed the origin URL (thus inheriting the force prefix).
The bug is not considered a security issue and the fix does not make
it harder for remote sites to intentionally circumvent blocks if
Privoxy isn't configured to enforce them.
Fixes #1695 reported by Korda.
- Normalize the request line in intercepted requests to make rewriting
the destination more convenient. Previously rewrites for intercepted
requests were expected to fail unless $hostport was being used, but
they failed "the wrong way" and would result in an out-of-memory
message (vanilla host patterns) or a crash (extended host patterns).
Reported by "Guybrush Threepwood" in #1694.
- Enable socket lingering for the correct socket.
Previously it was repeatedly enabled for the listen socket
instead of for the accepted socket. The bug was found by
code inspection and did not cause any (reported) issues.
- Detect and reject parameters for parameter-less actions.
Previously they were silently ignored.
- Fixed invalid reads in internal and outdated pcre code.
Found with afl-fuzz and AddressSanitizer.
- Prevent invalid read when loading invalid action files.
Found with afl-fuzz and AddressSanitizer.
- Windows build: Use the correct function to close the event handle.
It's unclear if this bug had a negative impact on Privoxy's behaviour.
Reported by Jarry Xu in #891.
- In case of invalid forward-socks5(t) directives, use the
correct directive name in the error messages. Previously they
referred to forward-socks4t failures.
Reported by Joel Verhagen in #889.
- General improvements:
- Set NO_DELAY flag for the accepting socket. This significantly reduces
the latency if the operating system is not configured to set the flag
by default. Reported by Johan Sintorn in #894.
- Allow to build with mingw x86_64. Submitted by Rustam Abdullaev in #135.
- Introduce the new forwarding type 'forward-webserver'.
Currently it is only supported by the forward-override{} action and
there's no config directive with the same name. The forwarding type
is similar to 'forward', but the request line only contains the path
instead of the complete URL.
- The CGI editor no longer treats 'standard.action' special.
Nowadays the official "standards" are part of default.action
and there's no obvious reason to disallow editing them through
the cgi editor anyway (if the user decided that the lack of
authentication isn't an issue in her environment).
- Improved error messages when rejecting intercepted requests
with unknown destination.
- A couple of log messages now include the number of active threads.
- Removed non-standard Proxy-Agent headers in HTTP snipplets
to make testing more convenient.
- Include the error code for pcre errors Privoxy does not recognize.
- Config directives with numerical arguments are checked more carefully.
- Privoxy's malloc() wrapper has been changed to prevent zero-size
allocations which should only occur as the result of bugs.
- Various cosmetic changes.
- Action file improvements:
- Unblock ".deutschlandradiokultur.de/".
Reported by u302320 in #924.
- Add two fast-redirect exceptions for "yandex.ru".
- Disable filter{banners-by-size} for ".plasmaservice.de/".
- Unblock "klikki.fi/adv/".
- Block requests for "resources.infolinks.com/".
Reported by "Black Rider" on ijbswa-users@.
- Block a bunch of criteo domains.
Reported by Black Rider.
- Block "abs.proxistore.com/abe/".
Reported by Black Rider.
- Disable filter{banners-by-size} for ".black-mosquito.org/".
- Disable fast-redirects for "disqus.com/".
- Documentation improvements:
- FAQ: Explicitly point fingers at ASUS as an example of a
company that has been reported to force malware based on
Privoxy upon its customers.
- Correctly document the action type for a bunch of "multi-value"
actions that were incorrectly documented to be "parameterized".
Reported by Gregory Seidman on ijbswa-users@.
- Fixed the documented type of the forward-override{} action
which is obviously 'parameterized'.
- Website improvements:
- Users who don't trust binaries served by SourceForge
can get them from a mirror. Migrating away from SourceForge
is planned for 2016 (TODO list item #53).
- The website is now available as onion service
(http://jvauzb4sb3bwlsnc.onion/).
*** Version 3.0.23 stable ***
- Bug fixes:
- Fixed a DoS issue in case of client requests with incorrect
chunk-encoded body. When compiled with assertions enabled
(the default) they could previously cause Privoxy to abort().
Reported by Matthew Daley. CVE-2015-1380.
- Fixed multiple segmentation faults and memory leaks in the
pcrs code. This fix also increases the chances that an invalid
pcrs command is rejected as such. Previously some invalid commands
would be loaded without error. Note that Privoxy's pcrs sources
(action and filter files) are considered trustworthy input and
should not be writable by untrusted third-parties. CVE-2015-1381.
- Fixed an 'invalid read' bug which could at least theoretically
cause Privoxy to crash. So far, no crashes have been observed.
CVE-2015-1382.
- Compiles with --disable-force again. Reported by Kai Raven.
- Client requests with body that can't be delivered no longer
cause pipelined requests behind them to be rejected as invalid.
Reported by Basil Hussain.
- General improvements:
- If a pcrs command is rejected as invalid, Privoxy now logs
the cause of the problem as text. Previously the pcrs error
code was logged.
- The tests are less likely to cause false positives.
- Action file improvements:
- '.sify.com/' is no longer blocked. Apparently it is not actually
a pure tracking site (anymore?). Reported by Andrew on ijbswa-users@.
- Unblock banners on .amnesty.de/ which aren't ads.
- Documentation improvements:
- The 'Would you like to donate?' section now also contains
a "Paypal" address.
- The list of supported operating systems has been updated.
- The existence of the SF support and feature trackers has been
deemphasized because they have been broken for months.
Most of the time the mailing lists still work.
- The claim that default.action updates are sometimes released
on their own has been removed. It hasn't happened in years.
- Explicitly mention that Tor's port may deviate from the default
when using a bundle. Requested by Andrew on ijbswa-users@.
*** Version 3.0.22 stable ***
- Bug fixes:
- Fixed a memory leak when rejecting client connections due to
the socket limit being reached (CID 66382). This affected
Privoxy 3.0.21 when compiled with IPv6 support (on most
platforms this is the default). CVE-2015-1030.
- Fixed an immediate-use-after-free bug (CID 66394) and two
additional unconfirmed use-after-free complaints made by
Coverity scan (CID 66391, CID 66376). CVE-2015-1031.
- Actually show the FORCE_PREFIX value on the show-status page.
- Properly deal with Keep-Alive headers with timeout= parameters
If the timeout still can't be parsed, use the configured
timeout instead of preventing the client from keeping the
connection alive. Fixes #3615312/#870 reported by Bernard Guillot.
- Not using any filter files no longer results in warning messages
unless an action file is referencing header taggers or filters.
Reported by Stefan Kurtz in #3614835.
- Fixed a bug that prevented Privoxy from reusing some reusable
connections. Two bit masks with different purpose unintentionally
shared the same bit.
- A couple of additional bugs were discovered by Coverity Scan.
The fixes that are not expected to affect users are not explicitly
mentioned here, for details please have a look at the CVS logs.
- General improvements:
- Introduced negative tag patterns NO-REQUEST-TAG and NO-RESPONSE-TAG.
They apply if no matching tag is found after parsing client or
server headers.
- Add support for external filters which allow to process the
response body with a script or program written in any language
the platform supports. External filters are enabled with
+external-filter{} after they have been defined in one of the
filter files with a header line starting with "EXTERNAL-FILTER:".
External filter support is experimental, not compiled by default
and known not to work on all platforms.
- Add support for the 'PATCH' method as defined in RFC5789.
- Reject requests with unsupported Expect header values.
Fixes a couple of Co-Advisor tests.
- Normalize the HTTP-version in forwarded requests and responses.
This is an explicit RFC 2616 MUST and RFC 7230 mandates that
intermediaries send their own HTTP-version in forwarded
messages.
- Server 'Keep-Alive' headers are no longer forwarded. From a user's
point of view it doesn't really matter, but RFC 2616 (obsolete)
mandates that the header is removed and this fixes a Co-Advisor
complaint.
- Change declared template file encoding to UTF-8. The templates
already used a subset of UTF-8 anyway and changing the declaration
allows to properly display UTF-8 characters used in the action files.
This change may require existing action files with ISO-8859-1
characters that aren't valid UTF-8 to be converted to UTF-8.
Requested by Sam Chen in #582.
- Do not pass rejected keep-alive timeouts to the server. It might
not have caused any problems (we know of), but doing the right
thing shouldn't hurt either.
- Let log_error() use its own buffer size #define to make changing
the log buffer size slightly less inconvenient.
- Turned single-threaded into a "proper" toggle directive with arguments.
- CGI templates no longer enforce new windows for some links.
- Remove an undocumented workaround ('HOST' header removal) for
an Apple iTunes bug that according to #729900 got fixed in 2003.
- Action file improvements:
- The pattern 'promotions.' is no longer being blocked.
Reported by rakista in #3608540.
- Disable fast-redirects for .microsofttranslator.com/.
- Disable filter{banners-by-size} for .dgb-tagungszentren.de/.
- Add adn.speedtest.net as a site-specific unblocker.
Support request #3612908.
- Disable filter{banners-by-size} for creativecommons.org/.
- Block requests to data.gosquared.com/. Reported by cbug in #3613653.
- Unblock .conrad./newsletter/. Reported by David Bo in #3614238.
- Unblock .bundestag.de/.
- Unblock .rote-hilfe.de/.
- Disable fast-redirects for .facebook.com/plugins/like.php.
- Unblock Stackexchange popup URLs that aren't used to serve ads.
Reported by David Wagner in #3615179.
- Disable fast-redirects for creativecommons.org/.
- Unblock .stopwatchingus.info/.
- Block requests for .adcash.com/script/.
Reported by Tyrexionibus in #3615289.
- Disable HTML filters if the response was tagged as JavaScript.
Filtering JavaScript code with filters intended to deal with HTML
is usually a waste of time and, more importantly, may break stuff.
- Use a custom redirect{} for .washingtonpost.com/wp-apps/imrs\.php\?src=
Previously enabling the 'Advanced' settings (or manually enabling
+fast-redirects{}) prevented some images from being loaded properly.
- Unblock "adina*." Fixes #919 reported by Morton A. Goldberg.
- Block '/.*DigiAd'.
- Unblock 'adele*.'. Reported by Adele Lime in #1663.
- Disable banners-by-size for kggp.de/.
- Filter file improvements & bug fixes:
- Decrease the chances that js-annoyances creates invalid JavaScript.
Submitted by John McGowan on ijbswa-users@.
- Let the msn filter hide 'related' ads again.
- Remove a stray '1' in the 'html-annoyances' filter.
- Prevent img-reorder from messing up img tags with empty src
attributes. Fixes #880 reported by Duncan.
- Documentation improvements:
- Updated the 'Would you like to donate?' section.
- Note that invalid forward-override{} parameter syntax isn't
detected until the parameter is used.
- Add another +redirect{} example: a shortcut for illumos bugs.
- Make it more obvious that many operating systems support log
rotation out of the box.
- Fixed dead links. Reported by Mark Nelson in #3614557.
- Rephrased the 'Why is the configuration so complicated?' answer
to be slightly less condescending. Anonymously suggested in #3615122.
- Be more explicit about accept-intercepted-requests's lack of MITM support.
- Make 'demoronizer' FAQ entries more generic.
- Add an example hostname to the --pre-chroot-nslookup description.
- Add an example for a host pattern that matches an IP address.
- Rename the 'domain pattern' to 'host pattern' as it may
contain IP addresses as well.
- Recommend forward-socks5t when using Tor. It seems to work fine and
modifying the Tor configuration to profit from it hasn't been necessary
for a while now.
- Add another redirect{} example to stress that redirect loops can
and should be avoided.
- The usual spelling and grammar fixes. Parts of them were
reported by Reuben Thomas in #3615276.
- Mention the PCRS option letters T and D in the filter section.
- Clarify that handle-as-empty-doc-returns-ok is still useful
and will not be removed without replacement.
- Note that security issues shouldn't be reported using the bug tracker.
- Clarify what Privoxy does if both +block{} and +redirect{} apply.
- Removed the obsolete bookmarklets section.
- Build system improvements:
- Let --with-group properly deal with secondary groups.
Patch submitted by Anatoly Arzhnikov in #3615187.
- Fix web-actions target.
- Add a web-faq target that only updates the FAQ on the webserver.
- Remove already-commented-out non-portable DOSFILTER alternatives.
- Remove the obsolete targets dok-put and dok-get.
- Add a sf-shell target.
*** Version 3.0.21 stable ***
- Bug fixes:
- On POSIX-like platforms, network sockets with file descriptor
values above FD_SETSIZE are properly rejected. Previously they
could cause memory corruption in configurations that allowed
the limit to be reached.
- Proxy authentication headers are removed unless the new directive
enable-proxy-authentication-forwarding is used. Forwarding the
headers potentially allows malicious sites to trick the user
into providing them with login information.
Reported by Chris John Riley.
- Compiles on OS/2 again now that unistd.h is only included
on platforms that have it.
- General improvements:
- The show-status page shows the FEATURE_STRPTIME_SANITY_CHECKS status.
- A couple of assert()s that could theoretically dereference
NULL pointers in debug builds have been relocated.
- Added an LSB info block to the generic start script.
Based on a patch from Natxo Asenjo.
- The max-client-connections default has been changed to 128
which should be more than enough for most setups.
- Action file improvements:
- Block rover.ebay./ar.*\&adtype= instead of "/.*\&adtype=" which
caused too man false positives.
Reported by u302320 in #360284, additional feedback from Adam Piggott.
- Unblock '.advrider.com/' and '/.*ADVrider'.
Anonymously reported in #3603636.
- Stop blocking '/js/slider\.js'.
Reported by Adam Piggott in #3606635 and _lvm in #2791160.
- Filter file improvements:
- Added an iframes filter.
- Documentation improvements:
- The whole GPLv2 text is included in the user manual now,
so Privoxy can serve it itself and the user can read it
without having to wade through GPLv3 ads first.
- Properly numbered and underlined a couple of section titles
in the config that where previously overlooked due to a flaw
in the conversion script. Reported by Ralf Jungblut.
- Improved the support instruction to hopefully make it harder to
unintentionally provide insufficient information when requesting
support. Previously it wasn't obvious that the information we need
in bug reports is usually also required in support requests.
- Removed documentation about packages that haven't been provided
in years.
- Privoxy-Regression-Test:
- Only log the test number when not running in verbose mode
The position of the test is rarely relevant and it previously
wasn't exactly obvious which one of the numbers was useful to
repeat the test with --test-number.
- GNUmakefile improvements:
- Factor generate-config-file out of config-file to make testing
more convenient.
- The clean target now also takes care of patch leftovers.
*** Version 3.0.20 beta ***
- Bug fixes:
- Client sockets are now properly shutdown and drained before being
closed. This fixes page truncation issues with clients that aggressively
pipeline data on platforms that otherwise discard already written data.
The issue mainly affected Opera users and was initially reported
by Kevin in #3464439, szotsaki provided additional information to track
down the cause.
- Fix latency calculation for shared connections (disabled by default).
It was broken since their introduction in 2009. The calculated latency
for most connections would be 0 in which case the timeout detection
failed to account for the real latency.
- Reject URLs with invalid port. Previously they were parsed incorrectly and
characters between the port number and the first slash were silently
dropped as shown by curl test 187.
- The default-server-timeout and socket-timeout directives accept 0 as
valid value.
- Fix a race condition on Windows that could cause Privoxy to become
unresponsive after toggling it on or off through the taskbar icon.
Reported by Tim H. in #3525694.
- Fix the compilation on Windows when configured without IPv6 support.
- Fix an assertion that could cause debug builds to abort() in case of
socks5 connection failures with "debug 2" enabled.
- Fix an assertion that could cause debug builds to abort() if a filter
contained nul bytes in the replacement text.
- General improvements:
- Significantly improved keep-alive support for both client and server
connections.
- New debug log level 65536 which logs all actions that were applied to
the request.
- New directive client-header-order to forward client headers in a
different order than the one in which they arrived.
- New directive tolerate-pipelining to allow client-side pipelining.
If enabled (3.0.20 beta enables it by default), Privoxy will keep
pipelined client requests around to deal with them once the current
request has been served.
- New --config-test option to let Privoxy exit after checking whether or not
the configuration seems valid. The limitations noted in TODO #22 and #23
still apply. Based on a patch by Ramkumar Chinchani.
- New limit-cookie-lifetime{} action to let cookies expire before the end
of the session. Suggested by Rick Sykes in #1049575.
- Increase the hard-coded maximum number of actions and filter files from
10 to 30 (each). It doesn't significantly affect Privoxy's memory usage
and recompiling wasn't an option for all Privoxy users that reached the
limit.
- Add support for chunk-encoded client request bodies. Previously
chunk-encoded request bodies weren't guaranteed to be forwarded correctly,
so this can also be considered a bug fix although chunk-encoded request
bodies aren't commonly used in the real world.
- Add support for Tor's optimistic-data SOCKS extension, which can reduce the
latency for requests on newly created connections. Currently only the
headers are sent optimistically and only if the client request has already
been read completely which rules out requests with large bodies.
- After preventing the client from pipelining, don't signal keep-alive
intentions. When looking at the response headers alone, it previously
wasn't obvious from the client's perspective that no additional responses
should be expected.
- Stop considering client sockets tainted after receiving a request with body.
It hasn't been necessary for a while now and unnecessarily causes test
failures when using curl's test suite.
- Allow HTTP/1.0 clients to signal interest in keep-alive through the
Proxy-Connection header. While such client are rare in the real world, it
doesn't hurt and couple of curl tests rely on it.
- Only remove duplicated Content-Type headers when filters are enabled.
If they are not it doesn't cause ill effects and the user might not want it.
Downgrade the removal message to LOG_LEVEL_HEADER to clarify that it's not
an error in Privoxy and is unlikely to cause any problems in general.
Anonymously reported in #3599335.
- Set the socket option SO_LINGER for the client socket.
- Move several variable declarations to the beginning of their code block.
It's required when compiling with gcc 2.95 which is still used on some
platforms. Initial patch submitted by Simon South in #3564815.
- Optionally try to sanity-check strptime() results before trusting them.
Broken strptime() implementations have caused problems in the past and
the most recent offender seems to be FreeBSD's libc (standards/173421).
- When filtering is enabled, let Range headers pass if the range starts at
the beginning. This should work around (or at least reduce) the video
playback issues with various Apple clients as reported by Duc in #3426305.
- Do not confuse a client hanging up with a connection time out. If a client
closes its side of the connection without sending a request line, do not
send the CLIENT_CONNECTION_TIMEOUT_RESPONSE, but report the condition
properly.
- Allow closing curly braces as part of action values as long as they are
escaped.
- On Windows, the logfile is now written before showing the GUI error
message which blocks until the user acknowledges it.
Reported by Adriaan in #3593603.
- Remove an unreasonable parameter limit in the CGI interface. The new
parameter limit depends on the memory available and is currently unlikely
to be reachable, due to other limits in both Privoxy and common clients.
Reported by Andrew on ijbswa-users@.
- Decrease the chances of parse failures after requests with unsupported
methods were sent to the CGI interface.
- Action file improvements:
- Remove the comment that indicated that updated default.action versions
are released on their own.
- Block 'optimize.indieclick.com/' and 'optimized-by.rubiconproject.com/'
- Unblock 'adjamblog.wordpress.com/' and 'adjamblog.files.wordpress.com/'.
Reported by Ryan Farmer in #3496116.
- Unblock '/.*Bugtracker'. Reported by pwhk in #3522341.
- Add test URLs for '.freebsd.org' and '.watson.org'.
- Unblock '.urbandictionary.com/popular'.
- Block '.adnxs.com/'.
- Block 'farm.plista.com/widgetdata.php'.
- Block 'rotation.linuxnewmedia.com/'.
- Block 'reklamy.sfd.pl/'. Reported by kacperdominik in #3399948.
- Block 'g.adspeed.net/'.
- Unblock 'websupport.wdc.com/'. Reported by Adam Piggot in #3577851.
- Block '/openx/www/delivery/'.
- Disable fast-redirects for '.googleapis.com/'.
- Block 'imp.double.net/'. Reported by David Bo in #3070411.
- Block 'gm-link.com/' which is used for email tracking.
Reported by David Bo in #1812733.
- Verify that requests to "bwp." are blocked. URL taken from #1736879
submitted by Francois Marier.
- Block '/.*bannerid='. Reported by Adam Piggott in #2975779.
- Block 'cltomedia.info/delivery/' and '.adexprt.com/'.
Anonymously reported in #2965254.
- Block 'de17a.com/'. Reported by David Bo in #3061472.
- Block 'oskar.tradera.com/'. Reported by David Bo in #3060596.
- Block '/scripts/webtrends\.js'. Reported by johnd16 in #3002729.
- Block requests for 'pool.*.adhese.com/'. Reported by johnd16 in #3002716.
- Update path pattern for Coremetrics and add tests.
Pattern and URLs submitted by Adam Piggott #3168443.
- Enable +fast-redirects{check-decoded-url} for 'tr.anp.se/'.
Reported by David Bo in #3268832.
- Unblock '.conrad.se/newsletter/banners/'. Reported by David Bo in #3413824.
- Block '.tynt.com/'. Reported by Dan Stahlke in #3421767.
- Unblock '.bbci.co.uk/radio/'. Reported by Adam Piggott in #3569603.
- Block requests to 'service.maxymiser.net/'.
Reported by johnd16 in #3118401 (with a previous URL).
- Disable fast-redirects for Google's "let's pretend your computer is
infected" page.
- Unblock '/.*download' to resolve actionsfile feedback #3498129.
Submitted by Steven Kolins (soundcloud.com not working).
- Unblock '.wlxrs.com/' which is required by hotmail.com.
Fixes #3413827 submitted by David Bo.
- Add two unblock patterns for popup radio and TV players.
Submitted by Adam Piggott in #3596089.
- Filter file improvements & bug fixes:
- Add a referer tagger.
- Reduce the likelihood that the google filter messes up HTML-generating
JavaScript. Reported by Zeno Kugy in #3520260.
- Documentation improvements:
- Revised all OS X sections due to new packaging module (OSXPackageBuilder).
- Update the list of supported operating systems to clarify that all Windows
versions after 95 are expected to work and note that the platform-specific
code for AmigaOS and QNX currently isn't maintained.
- Update 'Signals' section, the only explicitly handled signals are SIGINT,
SIGTERM and SIGHUP.
- Add Haiku to the list of operating systems on which Privoxy is known to
run.
- Add DragonFly to the list of BSDs on which Privoxy is known to run.
- Removed references to redhat-specific documentation set since it no longer
exists.
- Removed references to building PDFs since we no longer do so.
- Multiple listen-address directives are supported since 3.0.18, correct the
documentation to say so.
- Remove bogus section about long and short being preferable to int.
- Corrected some Internet JunkBuster references to Privoxy.
- Removed references to www.junkbusters.com since it is no longer
maintained. Reported by Angelina Matson.
- Various grammar and spelling corrections
- Add a client-header-tagger{} example for disabling filtering for range
requests.
- Correct a URL in the "Privoxy with Tor" FAQ.
- Spell 'refresh-tags' correctly. Reported by Don in #3571927.
- Sort manpage options alphabetically.
- Remove an incorrect sentence in the toggle section. The toggle state
doesn't affect whether or not the Windows version uses the tray icon.