Bug 656197

http://susepaste.org/860249
shows that prnt/hpijs/services.cpp logs zillions of same messages
in one same second like
---------------------------------------------------------------------
Nov 24 18:51:30 linux-icwg Deskjet-6940-series:
  prnt/hpijs/services.cpp 386: unable to write to output, fd=1,
  count=4096: Broken pipe
---------------------------------------------------------------------

See
https://bugzilla.novell.com/show_bug.cgi?id=557000#c2

There is already a matching upstream bug report:
https://bugs.launchpad.net/hplip/+bug/439550
and there should be the workaround:
Stop cups printer and restart cups printer,
then the job should be processed correctly.

*** This bug has been marked as a duplicate of bug 557000 ***

Please allow me.

Besides the printing problem, there is a problem with syslog: it did not filter out the thousands of repeated log entries till the system crashed. That is, IMO, a more important problem.

I'll change component to "basesystem" and reopen for re-consideration.


David:

Please add the result of this command:

grep SYSLOG_DAEMON /etc/sysconfig/syslog

FYI:

In the HPLIP sources in /prnt/hpijs/services.cpp there is the line
------------------------------------------------------------------------------
BUG("unable to write to output, fd=%d, count=%d: %m\n", OutputPath, *Count);
------------------------------------------------------------------------------

and "BUG" is defined in prnt/hpijs/bug.h (long line wrapped here):
------------------------------------------------------------------------------
#include <syslog.h>
#include <stdio.h>

#define _STRINGIZE(x) #x
#define STRINGIZE(x) _STRINGIZE(x)

#define BUG(args...) {syslog(LOG_ERR, __FILE__ " " STRINGIZE(__LINE__) ": "
args); \
fprintf(stderr, __FILE__ " " STRINGIZE(__LINE__) ": " args);}
------------------------------------------------------------------------------

By default we use syslog-ng (I don't know since which openSUSE version)
but syslog-ng does not filter out duplicates (what the old syslogd did).

This syslog-ng behaviour is not a bug in syslog-ng but at least
currently it is the intended behaviour of syslog-ng
which could be only changed by upstream syslog-ng.

(In reply to comment #4)
> By default we use syslog-ng (I don't know since which openSUSE version)
> but syslog-ng does not filter out duplicates (what the old syslogd did).

That is not entirely correct. 11.2 and 11.3 use rsyslog by default (I just checked three installs of 11.2 and one of 11.3). 11.0 used syslog-ng. 11.1 I don't know.

Please, lets wait till David can confirm what he has before closing this one.

David:

Please add the result of this command:

grep SYSLOG_DAEMON /etc/sysconfig/syslog

As workaround for the duplicate message problem, you have following options:

 - add a filter discarding the messsage to the syslog-ng.conf
 - install rsyslog (default) and set "$RepeatedMsgReduction on"
   in /etc/rsyslog.conf
 - install syslogd.

That the syslog-ng does not filter duplicates, is known upstream since
many many years. People ask for this feature on the mailing lists and
there are several patches for different syslog-ng versions (almost each
version is a complete rewrite) on the mailing lists [I wrote one for
1.6.x version as well], but the patches AFAIK never found their way
into the upstream versions.

https://lists.balabit.hu/pipermail/syslog-ng/2004-February/005817.html
https://lists.balabit.hu/pipermail/syslog-ng/2008-May/011698.html

So from syslog-ng side => Resolve as UPSTREAM.

Please check the suppress() option at
http://www.balabit.com/sites/default/files/documents/syslog-ng-ose-v3.1-guide-admin-en.html/reference_destination_file.html
It is not used as default, but available.

Ah... this means, it has been implemented in 3.1 after all.
But there is a 2.0.9 on 11.2...

(In reply to comment #7)
> David:
> 
> Please add the result of this command:
> 
> grep SYSLOG_DAEMON /etc/sysconfig/syslog

I'm not sure if you got this or not,
The return was this
> > SYSLOG_DAEMON="rsyslogd"

Thanks David. Yes, that's what I thought. Changing the tittle accordingly.

WAS: Bug 656197 - syslog-ng does not filter that hpijs/services.cpp fills up log files with thousands of same messages

NEW: Bug 656197 - rsyslogd does not filter that hpijs/services.cpp fills up log files with thousands of same messages


P.S: I see this was assigned to me. I'm not the maintainer, just a helper trying to get the correct information. Reverting to defaults.

This is a bug either in rsyslog or in its openSUSE configuration

Thanks Carlos, David!

(In reply to comment #12)
> This is a bug either in rsyslog or in its openSUSE configuration

It is neither a bug in rsyslog, nor in openSUSE, but as already written in
comment 8, it is a configration option and openSUSE is using the rsyslogd
default setting of "$RepeatedMsgReduction off".

I'm quite sure, somebody will report the change as bug [that messages dups
are dropped], but I've changed to explicit "on" in Base:System / Factory,
because it useful to catch amok running programs logging exactly same log
message...

But this IMO does not change the initial resolution as UPSTREAM: the bug is
in hpijs/services.cpp.

Mr Maintenance,

do we need to provide update packages with a change of /etc/rsyslog.conf
to use:
  "$RepeatedMsgReduction on"
as default? Another Objections/Instructions?

> It is neither a bug in rsyslog, nor in openSUSE, but as already written in
> comment 8, it is a configration option and openSUSE is using the rsyslogd
> default setting of "$RepeatedMsgReduction off".

Question:

When a message is repeated many times and filtered out, doesn't rsyslog print a message of the type "skipped 100 identical messages", or something like that? I have seen that on openSUSE logs, time ago...

If that is not so, I think it would be useful perhaps to leave messages on in factory, and filter them in the stable releases.


As to the bug being solely the fault of hpijs/services.cpp, I mildly object: a faulty program should not crash a system (disk full).

Only FYI:

Some days ago at home our file system where /home/ is mounted
was filled up by Gimp when it crashed while processing a huge
image file which resulted an incredible huge file in Gimp's
directory where Gimp stores its internal (temporary) stuff.

Almost any (faulty) program can fill up the disk. E.g. like
----------------------------------------------------------------
# cat /dev/urandom >fillup_file
----------------------------------------------------------------
Of course some space is reserved on "/" for root
and one could set up disk quotas and so on...

(In reply to comment #15)
> > It is neither a bug in rsyslog, nor in openSUSE, but as already written in
> > comment 8, it is a configration option and openSUSE is using the rsyslogd
> > default setting of "$RepeatedMsgReduction off".
> 
> Question:
> 
> When a message is repeated many times and filtered out, doesn't rsyslog
> print a message of the type "skipped 100 identical messages", or something
> like that? I have seen that on openSUSE logs, time ago...

As I already wrote:
 - syslogd does this by default.
 - for syslog-ng-1.6 I wrote a patch to do it by default as well.
   in syslog-ng-2.0.9, there is no suppress() option doing it as in syslog-ng-3.1.
 - in rsyslogd it is a option with built-in default 'off'.

For openSUSE-11.4 I've enabled the RepeatedMsgReduction option and I'll
enable the suppress() option for syslog-ng as well in the next submit to
catch this special case.


Because it is an option in rsyslog (=default), I've asked in comment 14,
whether I have to provide an update package with new /etc/rsyslog.conf,
that enables this option or not.


> If that is not so, I think it would be useful perhaps to leave messages
> on in factory, and filter them in the stable releases.
>
> As to the bug being solely the fault of hpijs/services.cpp, I mildly object:
> a faulty program should not crash a system (disk full).

It is not possible to prohibit this.

The option catches a special case only:

   programs that log *exactly* same message.

As soon as a message differs even one bit, the message gets logged.
So a user or program just need to run e.g. the following to fill up
the log-disk:

i=0 ; while true ; do logger "======== $((i++)) ==========" ; done

==>> Best is to always put /var/log to a separate disk.

(In reply to comment #17)
>  - for syslog-ng-1.6 I wrote a patch to do it by default as well.
>    in syslog-ng-2.0.9, there is no suppress() option doing it as in
>    syslog-ng-3.1.

BTW: Also here: suppress() is an option in 3.1 and 'off' by default.

> Because it is an option in rsyslog (=default), I've asked in comment 14,
> whether I have to provide an update package with new /etc/rsyslog.conf,
> that enables this option or not.

IMHO we should only change the behavior in the next openSUSE release (-1).

So no update here. Be so kind and fix it with the next openSUSE release.

fixed in factory for 11.4.

This is an autogenerated message for OBS integration:
This bug (656197) was mentioned in
https://build.opensuse.org/request/show/672696 15.1 / syslog-ng