Acknowledgement sent
to Holger Levsen <[email protected]>:
New Bug report received and forwarded. Copy sent to Debian logcheck Team <[email protected]>.
(Thu, 16 Mar 2023 12:21:03 GMT) (full text, mbox, link).
Package: logcheck
Version: 1.4.2
Severity: wishlist
Dear Maintainer,
since bookworm rsyslog defaults to timestamps in short-iso-precise format,
while logcheck rules (and journald) still default to the old rule format,
and while the default logcheck rules in the package are easily switched,
this poses problems for larger installations with local logcheck rules
used on systems running different suites.
So I'd recommend to add some advice to logcheck/debian/NEWS based on this
conversation on #debian-devel just now:
<h01ger> mbiebl: given #475303 as context, what's your advice on resolving this: rsyslog now uses new time format, journald uses the old format and logcheck rules are in the old format. does journald use the old format because this is bookworm upgraded and not fresh install? should i simply remove/ignore the timestamps in my logcheck rules? should we add some hint to the release notes?
- zwiebelbot- | (#debian-devel) Debian#475303: Enable support for high precision timestamps - https://bugs.debian.org/475303
<mbiebl> | h01ger: I wasn't aware that logcheck checks the journal until 2 weeks ago someone asked about it
<mbiebl> https://github.com/systemd/systemd/issues/26639 was the result of this discussion
<h01ger> yeah, its a new feature (and sensible! i want it!)
<h01ger> mbiebl: issues/26639 seems sensible too. will/shall that land in bookworm?
<mbiebl> atm, it doesn't look like
<h01ger> dropping timestamps from all logcheck rules could migate this and is an easy way to run mixed suite setups
<h01ger> though it makes me wonder why i kept those for the last 10 or so years, if they now suddenly are not needed ;)
<h01ger> breaking habbits..
<mbiebl> maybe you could make the existing parsing/regexps work with both formats
<mbiebl> 2023-03-16T12:45:45.159206+01:00
<mbiebl> vs
<mbiebl> 2023-03-16T12:50:13.503482+0100
<mbiebl> you'd basically just need an optional ':' in the timezone information
<mbiebl> that is rsyslog and journalctl --output=short-iso-precise
<h01ger> doesnt help with systems not yet running bookworm.
<h01ger> (and those are not all running bullseye either, but older releases too)
<mbiebl> I thought this was about fixing it in bookworm
<h01ger> well, its also about using logcheck for all 'my' systems. i (co-)maintain several setups using logcheck...
<h01ger> and i'm sure i'm not the only one who'll encounter this
<h01ger> since when do both rsyslog and journalctl support --output=short-iso-precise ?
<h01ger> #475303 is from 2008, so i assume changing rsyslog format for old systems could work
<mbiebl> rsyslog uses rfc 3339 by default since bookworm (has supported for 10+years), journald supports short-iso-precise since I can reemember
<h01ger> cool, so i'll switch to short-iso-precise everywhere at once
<mbiebl> systemd, just checked: since v234
<h01ger> i guess this could be a NEWS entry for logcheck
<mbiebl> | h01ger: so you'd miss o-o-stable (v232)
<h01ger> mbiebl: can i put this conversation in a wishlist bug against logcheck, asking to document this in NEWS?
<mbiebl> It was my impression that logcheck changed the regexps which match the timestamps in a way that both matched the old and new format?
<mbiebl> sure, feel free to quote this wherever you like
<h01ger> mbiebl: everyone using logcheck has local rules which need to be changed
<h01ger> hmpf, one setup has 13 machines still running stretch...
<h01ger> mbiebl: & thanks! ("feel free..")
<mbiebl> we do provide backports fwiw
<mbiebl> not sure if that is option in your case
<h01ger> it is, thanks!
<h01ger> systemd | 241-5~bpo9+1 | stretch-backports
<h01ger> cool cool. happy this is conceptually solved now ;) i've migrated very few systems to bookworm yet and have been noticing those 1-2 new daily mails since 2 weeks or so, knowing this will need solving eventually...
<h01ger> mbiebl: thank you very much for this conversation!
--
cheers,
Holger
⢀⣴⠾⠻⢶⣦⠀
⣾⠁⢠⠒⠀⣿⡁ holger@(debian|reproducible-builds|layer-acht).org
⢿⡄⠘⠷⠚⠋⠀ OpenPGP: B8BF54137B09D35CF026FE9D 091AB856069AAA1C
⠈⠳⣄
The pandemic isn’t over. We just stopped caring about other people.
Acknowledgement sent
to Holger Levsen <[email protected]>:
Extra info received and forwarded to list. Copy sent to Debian logcheck Team <[email protected]>.
(Thu, 16 Mar 2023 12:51:02 GMT) (full text, mbox, link).
<mbiebl> | h01ger: thanks for the bug report, but something got mixed up there: rsyslog uses rfc3339, not short-iso-precise
<mbiebl> short-iso-precise is an output format of journalctl that is similar to rfc3339 but differs in the presentation of the timezone information
<h01ger> *g*, thanks
<h01ger> mbiebl: this is also why you said this:
<h01ger> [12:51] < mbiebl> maybe you could make the existing parsing/regexps work with both formats
<h01ger> (above)
<mbiebl> yes
--
cheers,
Holger
⢀⣴⠾⠻⢶⣦⠀
⣾⠁⢠⠒⠀⣿⡁ holger@(debian|reproducible-builds|layer-acht).org
⢿⡄⠘⠷⠚⠋⠀ OpenPGP: B8BF54137B09D35CF026FE9D 091AB856069AAA1C
⠈⠳⣄
Fischers Fritz fischt Plastik.
Acknowledgement sent
to Holger Levsen <[email protected]>:
Extra info received and forwarded to list. Copy sent to Debian logcheck Team <[email protected]>.
(Thu, 16 Mar 2023 12:51:04 GMT) (full text, mbox, link).
<mbiebl> a properly placed [:]? might be all that's need
--
cheers,
Holger
⢀⣴⠾⠻⢶⣦⠀
⣾⠁⢠⠒⠀⣿⡁ holger@(debian|reproducible-builds|layer-acht).org
⢿⡄⠘⠷⠚⠋⠀ OpenPGP: B8BF54137B09D35CF026FE9D 091AB856069AAA1C
⠈⠳⣄
All data, over time, approaches deleted, or public. (@quinnnorton)
Acknowledgement sent
to Holger Levsen <[email protected]>:
Extra info received and forwarded to list. Copy sent to Debian logcheck Team <[email protected]>.
(Thu, 16 Mar 2023 21:36:10 GMT) (full text, mbox, link).
On Thu, Mar 16, 2023 at 05:52:44PM +0000, Richard Lewis wrote:
> Is it not the first entry, from version 1.4.0 from dec 2022 in
> /usr/share/doc/logcheck-database/README.logcheck-database.gz ??
aaaaaaaaaaah, thanks! I only checked /usr/share/doc/logcheck/NEWS.Debian.gz
but not /usr/share/doc/logcheck-database/NEWS.Debian.gz
> While I can sort of see an argument for putting this in logcheck's news
> instead (or as well) that doesnt seem correct to me...logcheck-database is
> what provides the rules for normal users - it is recommended by logcheck.
> I would assume people not using it know what they are doing.
>
> If you really want to catch all users shouldnt it be in rsyslog's
> NEWS.Debian ?
no, because it also effects logcheck users not using logcheck-database
at all. ;)
> What do you think the best way forward is?
>
> (I do intend to write something for debian's release notes about the
> rsyslog change, if no-one else does.)
that's great, maybe that's the best way forward indeed.
so maybe reassign this bug to src:release-notes?
> The wider issue is that logcheck has not been a package that works out of
> the box without significant configuration and has had minimal attention for
> several debian releases. we are trying to change that, but please give us
> some time while we understand the gap - i think debian is slightly
> fortunate to be releasing bookworm with a logcheck package that works at all
;) I very much appreciate your efforts bringing logcheck back in shape!
> I suspect most of the rules in debian are so old they never match anything,
> and there are definitely many updates needed. but i dont think anyone has
> the desire to do so before bookworm.
*g*
> i personally dont think it is worth even contemplating that work until we
> have revised the way rules are selected and the format they use.
I trust your judgement.
Thanks for maintaining logcheck.
--
cheers,
Holger
⢀⣴⠾⠻⢶⣦⠀
⣾⠁⢠⠒⠀⣿⡁ holger@(debian|reproducible-builds|layer-acht).org
⢿⡄⠘⠷⠚⠋⠀ OpenPGP: B8BF54137B09D35CF026FE9D 091AB856069AAA1C
⠈⠳⣄
In a world where you can be anything, be kind.
Acknowledgement sent
to Richard Lewis <[email protected]>:
Extra info received and forwarded to list. Copy sent to Debian logcheck Team <[email protected]>.
(Thu, 16 Mar 2023 21:57:09 GMT) (full text, mbox, link).
On Thu, 16 Mar 2023, 13:40 Holger Levsen, <[email protected]> wrote:
> On Thu, Mar 16, 2023 at 01:31:37PM +0000, Richard Lewis wrote:
> > I dont understand - logcheck rules cater for both formats since 1.4.1
> iirc
> > and this is already explained in NEWS.Debian. (and i thought that
> included
> > instructions for updating local rules in that)
>
> it's not. just checked the NEWS from 1.4.2 and it only explains
> that systemd's journal is now also checked. no word about different time
> formats.
>
Is it not the first entry, from version 1.4.0 from dec 2022 in
/usr/share/doc/logcheck-database/README.logcheck-database.gz ??
at least on my system it is there. i think my version is non-standard
(systemd unit is coming for trixie)
While I can sort of see an argument for putting this in logcheck's news
instead (or as well) that doesnt seem correct to me...logcheck-database is
what provides the rules for normal users - it is recommended by logcheck.
I would assume people not using it know what they are doing.
If you really want to catch all users shouldnt it be in rsyslog's
NEWS.Debian ?
What do you think the best way forward is?
(I do intend to write something for debian's release notes about the
rsyslog change, if no-one else does.)
The wider issue is that logcheck has not been a package that works out of
the box without significant configuration and has had minimal attention for
several debian releases. we are trying to change that, but please give us
some time while we understand the gap - i think debian is slightly
fortunate to be releasing bookworm with a logcheck package that works at all
I suspect most of the rules in debian are so old they never match anything,
and there are definitely many updates needed. but i dont think anyone has
the desire to do so before bookworm.
i personally dont think it is worth even contemplating that work until we
have revised the way rules are selected and the format they use.
Acknowledgement sent
to Holger Levsen <[email protected]>:
Extra info received and forwarded to list. Copy sent to Debian logcheck Team <[email protected]>.
(Thu, 16 Mar 2023 22:03:04 GMT) (full text, mbox, link).
On Thu, Mar 16, 2023 at 01:31:37PM +0000, Richard Lewis wrote:
> I dont understand - logcheck rules cater for both formats since 1.4.1 iirc
> and this is already explained in NEWS.Debian. (and i thought that included
> instructions for updating local rules in that)
it's not. just checked the NEWS from 1.4.2 and it only explains
that systemd's journal is now also checked. no word about different time formats.
> Did you maybe not upgade logcheck-database to latest version?
this is also about giving advice what to do with *local* logcheck rules.
> the longer term solution is perhaps macros in rules, which may happen in
> trixie.
we need some solution/workaround/configuration/advice for bookworm, else
people will just not use logcheck, if it creates too much noise.
> > <h01ger> dropping timestamps from all logcheck rules could migate this and
> > is an easy way to run mixed suite setups
> not sure the package should drop the prefixes,
i'm basically wondering why to have timestamp regexes in logcheck rules
at all. *not* having them has two benefits, I guess: a.) (very) slightly faster,
b.) easier to maintain/read by humans. what benefits *does* having them?
> > <mbiebl> It was my impression that logcheck changed the regexps which
> > match the timestamps in a way that both matched the old and new format?
> yes!
cool. so this sounds like easy advice to give for updating local rules! :)
and that's what I'm asking for to be done in debian/NEWS.
--
cheers,
Holger
⢀⣴⠾⠻⢶⣦⠀
⣾⠁⢠⠒⠀⣿⡁ holger@(debian|reproducible-builds|layer-acht).org
⢿⡄⠘⠷⠚⠋⠀ OpenPGP: B8BF54137B09D35CF026FE9D 091AB856069AAA1C
⠈⠳⣄
We live in a world where teenagers get more and more desperate trying to
convince adults to behave like grown ups.
Acknowledgement sent
to Richard Lewis <[email protected]>:
Extra info received and forwarded to list. Copy sent to Debian logcheck Team <[email protected]>.
(Thu, 16 Mar 2023 22:27:02 GMT) (full text, mbox, link).
On Thu, 16 Mar 2023, 12:21 Holger Levsen, <[email protected]> wrote:
>
>
> since bookworm rsyslog defaults to timestamps in short-iso-precise format,
> while logcheck rules (and journald) still default to the old rule format,
>
I dont understand - logcheck rules cater for both formats since 1.4.1 iirc
and this is already explained in NEWS.Debian. (and i thought that included
instructions for updating local rules in that)
can you clarify what the request for logcheck is here?
Did you maybe not upgade logcheck-database to latest version?
and while the default logcheck rules in the package are easily switched,
> this poses problems for larger installations with local logcheck rules
> used on systems running different suites.
>
the longer term solution is perhaps macros in rules, which may happen in
trixie. then rules can start
^@TIMESTAMP @HOSTNAME:.....$
(or whatever syntax is chosen)
and you could set TIMESTAMP to whatever you liked....
> <mbiebl> | h01ger: I wasn't aware that logcheck checks the journal until 2
> weeks ago someone asked about it
> <mbiebl> https://github.com/systemd/systemd/issues/26639 was the result
> of this discussion
> <h01ger> yeah, its a new feature (and sensible! i want it!)
>
it's actually not a new feature, was possible in at least bulleye, just
enabled it by default recently given the downgrade of rsyslog
<h01ger> mbiebl: issues/26639 seems sensible too. will/shall that land in
> bookworm?
> <mbiebl> atm, it doesn't look like
> <h01ger> dropping timestamps from all logcheck rules could migate this and
> is an easy way to run mixed suite setups
>
not sure the package should drop the prefixes,
<h01ger> though it makes me wonder why i kept those for the last 10 or so
> years, if they now suddenly are not needed ;)
> <h01ger> breaking habbits..
> <mbiebl> maybe you could make the existing parsing/regexps work with both
> formats
> <mbiebl> 2023-03-16T12:45:45.159206+01:00
> <mbiebl> vs
> <mbiebl> 2023-03-16T12:50:13.503482+0100
> <mbiebl> you'd basically just need an optional ':' in the timezone
> information
> <mbiebl> that is rsyslog and journalctl --output=short-iso-precise
> <h01ger> doesnt help with systems not yet running bookworm.
> <h01ger> (and those are not all running bullseye either, but older
> releases too)
> <mbiebl> I thought this was about fixing it in bookworm
> <h01ger> well, its also about using logcheck for all 'my' systems. i
> (co-)maintain several setups using logcheck...
> <h01ger> and i'm sure i'm not the only one who'll encounter this
> <h01ger> since when do both rsyslog and journalctl support
> --output=short-iso-precise ?
> <h01ger> #475303 is from 2008, so i assume changing rsyslog format for old
> systems could work
> <mbiebl> rsyslog uses rfc 3339 by default since bookworm (has supported
> for 10+years), journald supports short-iso-precise since I can reemember
> <h01ger> cool, so i'll switch to short-iso-precise everywhere at once
> <mbiebl> systemd, just checked: since v234
> <h01ger> i guess this could be a NEWS entry for logcheck
> <mbiebl> | h01ger: so you'd miss o-o-stable (v232)
> <h01ger> mbiebl: can i put this conversation in a wishlist bug against
> logcheck, asking to document this in NEWS?
> <mbiebl> It was my impression that logcheck changed the regexps which
> match the timestamps in a way that both matched the old and new format?
yes!
>
Acknowledgement sent
to Richard Lewis <[email protected]>:
Extra info received and forwarded to list. Copy sent to Debian logcheck Team <[email protected]>.
(Thu, 16 Mar 2023 22:27:07 GMT) (full text, mbox, link).
Acknowledgement sent
to Holger Levsen <[email protected]>:
Extra info received and forwarded to list. Copy sent to Debian logcheck Team <[email protected]>.
(Sat, 18 Mar 2023 15:12:02 GMT) (full text, mbox, link).
On Thu, Mar 16, 2023 at 06:00:06PM +0000, Holger Levsen wrote:
> aaaaaaaaaaah, thanks! I only checked /usr/share/doc/logcheck/NEWS.Debian.gz
> but not /usr/share/doc/logcheck-database/NEWS.Debian.gz
now that I read it and followed the advice and the very nice
sed example there, I can they that it worked flawlessly and was
very easy to do. Thank you for that NEWS entry!
> so maybe reassign this bug to src:release-notes?
this question is still open... though maybe cloning the bug is even
better, I'd really appreciated a small pointer to logcheck-database's NEWS
file in the NEWS for logcheck...
--
cheers,
Holger
⢀⣴⠾⠻⢶⣦⠀
⣾⠁⢠⠒⠀⣿⡁ holger@(debian|reproducible-builds|layer-acht).org
⢿⡄⠘⠷⠚⠋⠀ OpenPGP: B8BF54137B09D35CF026FE9D 091AB856069AAA1C
⠈⠳⣄
Bottled water companies don't produce water, they produce plastic bottles.
Acknowledgement sent
to Richard Lewis <[email protected]>:
Extra info received and forwarded to list. Copy sent to Debian logcheck Team <[email protected]>.
(Sat, 18 Mar 2023 18:57:02 GMT) (full text, mbox, link).
On Sat, 18 Mar 2023, 15:12 Holger Levsen, <[email protected]> wrote:
> On Thu, Mar 16, 2023 at 06:00:06PM +0000, Holger Levsen wrote:
> > aaaaaaaaaaah, thanks! I only checked
> /usr/share/doc/logcheck/NEWS.Debian.gz
> > but not /usr/share/doc/logcheck-database/NEWS.Debian.gz
>
> now that I read it and followed the advice and the very nice
> sed example there, I can they that it worked flawlessly and was
> very easy to do. Thank you for that NEWS entry!
>
> > so maybe reassign this bug to src:release-notes?
>
> this question is still open... though maybe cloning the bug is even
> better, I'd really appreciated a small pointer to logcheck-database's NEWS
> file in the NEWS for logcheck...
>
I have submitted something against release-notes so that is in hand.
rsyslog has #1031827 which seems to at least have had a response in 2023
I dont mind adding an entry for logcheck's NEWS as well as/instead of
logcheck-database's NEWS, @Mathias Gibbens what do you think?
The one drawback i see is that 99.9% of people will upgrade both logcheck
and logcheck-database together so will get 2 emails from apt-listchanges if
we put it in both..... So we should delete it from logcheck-database's NEWS
I think? - logcheck does require the same layout of rules even if you dont
use logcheck-database so i think this makes sense. I hope this would not
crash apt-listchanges fir unstable users if the NEWS file
shrinks/disappears due to whatever culls.old entries...?
Acknowledgement sent
to Richard Lewis <[email protected]>:
Extra info received and forwarded to list. Copy sent to Debian logcheck Team <[email protected]>.
(Sun, 12 May 2024 14:48:03 GMT) (full text, mbox, link).
Subject: Re: Bug#1033059: logcheck: NEWS advice how to deal with timestamps in
different formats
Date: Sun, 12 May 2024 15:44:56 +0100
On Sat, 18 Mar 2023 18:55:25 +0000 Richard Lewis
<[email protected]> wrote:
> On Sat, 18 Mar 2023, 15:12 Holger Levsen, <[email protected]> wrote:
>
> > On Thu, Mar 16, 2023 at 06:00:06PM +0000, Holger Levsen wrote:
> > > aaaaaaaaaaah, thanks! I only checked
> > /usr/share/doc/logcheck/NEWS.Debian.gz
> > > but not /usr/share/doc/logcheck-database/NEWS.Debian.gz
> >
> > now that I read it and followed the advice and the very nice
> > sed example there, I can they that it worked flawlessly and was
> > very easy to do. Thank you for that NEWS entry!
> >
> > > so maybe reassign this bug to src:release-notes?
> >
> > this question is still open... though maybe cloning the bug is even
> > better, I'd really appreciated a small pointer to logcheck-database's NEWS
> > file in the NEWS for logcheck...
Think we may as well close this bug, unless anyone objects. bookworm
release-notes cover the issue.
Next big change im planning to document in logcheck's NEWS.Debian to
catch all users - watch this space!
Debbugs is free software and licensed under the terms of the GNU General
Public License version 2. The current version can be obtained
from https://bugs.debian.org/debbugs-source/.