Attachment 713752 Details for Bug 1021307

'perf' output before the crash occured

2017-02-10_perfout.txt (text/plain), 7.62 KB, created by Siegbert Sadowski on 2017-02-10 21:23:15 UTC

(hide)

Description:

Filename:

MIME Type:

Creator: Siegbert Sadowski

Created: 2017-02-10 21:23:15 UTC

Size: 7.62 KB

patch

obsolete

>--- screenshot of perf trace -e --duration=1000 -e fdatasync,fsync,msync,sync,sync_file_range,syncfs --pid=$(pgrep systemd-journal) > perfout.txt ---
>
>serv2:~ # perf trace -e --duration=1000 -e fdatasync,fsync,msync,sync,sync_file_range,syncfs --pid=$(pgrep systemd-journal) > perfout.txt
>     0.009 ( 0.009 ms): fsync(fd: 21</var/log/journal/565b8fb65ab16a395a2a210a586a86d8/system.journal>) = 0
>     7.301 ( 7.267 ms): fsync(fd: 21</var/log/journal/565b8fb65ab16a395a2a210a586a86d8/system.journal>) = 0
>143808.034 ( 5.491 ms): fsync(fd: 21</var/log/journal/565b8fb65ab16a395a2a210a586a86d8/system.journal>) = 0
>315385.882 ( 5.869 ms): fsync(fd: 62                                                          ) = 0
>315385.912 ( 0.012 ms): fsync(fd: 62                                                          ) = 0
>444039.863 ( 0.009 ms): fsync(fd: 21</var/log/journal/565b8fb65ab16a395a2a210a586a86d8/system.journal>) = 0
>444047.092 ( 7.202 ms): fsync(fd: 21</var/log/journal/565b8fb65ab16a395a2a210a586a86d8/system.journal>) = 0
>444047.100 ( 0.003 ms): fsync(fd: 62                                                          ) = 0
>444052.954 ( 5.841 ms): fsync(fd: 62                                                          ) = 0
>527462.422 ( 6.762 ms): fsync(fd: 21</var/log/journal/565b8fb65ab16a395a2a210a586a86d8/system.journal>) = 0
>827574.318 ( 0.009 ms): fsync(fd: 21</var/log/journal/565b8fb65ab16a395a2a210a586a86d8/system.journal>) = 0
>827581.832 ( 7.487 ms): fsync(fd: 21</var/log/journal/565b8fb65ab16a395a2a210a586a86d8/system.journal>) = 0
>923174.728 ( 6.864 ms): fsync(fd: 21</var/log/journal/565b8fb65ab16a395a2a210a586a86d8/system.journal>) = 0
>1223366.966 ( 6.863 ms): fsync(fd: 21</var/log/journal/565b8fb65ab16a395a2a210a586a86d8/system.journal>) = 0
>1223372.772 ( 5.784 ms): fsync(fd: 21</var/log/journal/565b8fb65ab16a395a2a210a586a86d8/system.journal>) = 0
>1324767.427 ( 7.584 ms): fsync(fd: 21</var/log/journal/565b8fb65ab16a395a2a210a586a86d8/system.journal>) = 0
>1624896.290 ( 0.009 ms): fsync(fd: 21</var/log/journal/565b8fb65ab16a395a2a210a586a86d8/system.journal>) = 0
>1624902.948 ( 6.631 ms): fsync(fd: 21</var/log/journal/565b8fb65ab16a395a2a210a586a86d8/system.journal>) = 0
>1874086.505 ( 6.976 ms): fsync(fd: 21</var/log/journal/565b8fb65ab16a395a2a210a586a86d8/system.journal>) = 0
>2174204.226 ( 8.487 ms): fsync(fd: 21</var/log/journal/565b8fb65ab16a395a2a210a586a86d8/system.journal>) = 0
>2174209.658 ( 5.376 ms): fsync(fd: 21</var/log/journal/565b8fb65ab16a395a2a210a586a86d8/system.journal>) = 0
>2723550.207 ( 7.670 ms): fsync(fd: 21</var/log/journal/565b8fb65ab16a395a2a210a586a86d8/system.journal>) = 0
>3023771.689 ( 0.010 ms): fsync(fd: 21</var/log/journal/565b8fb65ab16a395a2a210a586a86d8/system.journal>) = 0
>3023779.164 ( 7.445 ms): fsync(fd: 21</var/log/journal/565b8fb65ab16a395a2a210a586a86d8/system.journal>) = 0
>3299910.512 ( 5.791 ms): fsync(fd: 21</var/log/journal/565b8fb65ab16a395a2a210a586a86d8/system.journal>) = 0
>1874086.505 ( 6.976 ms): fsync(fd: 21</var/log/journal/565b8fb65ab16a395a2a210a586a86d8/system.journal>) = 0
>2174204.226 ( 8.487 ms): fsync(fd: 21</var/log/journal/565b8fb65ab16a395a2a210a586a86d8/system.journal>) = 0
>2174209.658 ( 5.376 ms): fsync(fd: 21</var/log/journal/565b8fb65ab16a395a2a210a586a86d8/system.journal>) = 0
>2723550.207 ( 7.670 ms): fsync(fd: 21</var/log/journal/565b8fb65ab16a395a2a210a586a86d8/system.journal>) = 0
>3023771.689 ( 0.010 ms): fsync(fd: 21</var/log/journal/565b8fb65ab16a395a2a210a586a86d8/system.journal>) = 0
>3023779.164 ( 7.445 ms): fsync(fd: 21</var/log/journal/565b8fb65ab16a395a2a210a586a86d8/system.journal>) = 0
>3299910.512 ( 5.791 ms): fsync(fd: 21</var/log/journal/565b8fb65ab16a395a2a210a586a86d8/system.journal>) = 0
>3600073.579 ( 0.019 ms): fsync(fd: 21</var/log/journal/565b8fb65ab16a395a2a210a586a86d8/system.journal>) = 0
>3600081.126 ( 7.496 ms): fsync(fd: 21</var/log/journal/565b8fb65ab16a395a2a210a586a86d8/system.journal>) = 0
>3704607.611 ( 7.017 ms): fsync(fd: 21</var/log/journal/565b8fb65ab16a395a2a210a586a86d8/system.journal>) = 0
>4046200.367 (41340.416 ms): fsync(fd: 21</var/log/journal/565b8fb65ab16a395a2a210a586a86d8/system.journal>) = 0
>4046205.996 ( 5.600 ms): fsync(fd: 21</var/log/journal/565b8fb65ab16a395a2a210a586a86d8/system.journal>) = 0
>serv2:~ # -------
>If '-------' is not a typo you can use command-not-found to lookup the package that contains it, like this:
>    cnf -------
>serv2:~ # 
>serv2:~ # 
>
>
>---------- screenshot of journalctl -ef at point of crash ---------------
>Feb 10 20:05:52 serv2 systemd-journald[28584]: Journal started
>Feb 10 19:59:48 serv2 named[1874]: error (address not available) resolving './NS/IN': 2001:7fd::1#53
>Feb 10 19:59:57 serv2 systemd[1]: systemd-journald.service: Watchdog timeout (limit 3min)!
>Feb 10 19:59:48 serv2 named[1874]: error (address not available) resolving './NS/IN': 2001:500:2d::d#53
>Feb 10 19:59:48 serv2 named[1874]: error (address not available) resolving './NS/IN': 2001:dc3::35#53
>Feb 10 19:59:48 serv2 named[1874]: error (address not available) resolving './NS/IN': 2001:500:2f::f#53
>Feb 10 20:00:01 serv2 cron[28573]: pam_unix(crond:session): session opened for user root by (uid=0)
>Feb 10 20:04:04 serv2 CRON[28573]: pam_unix(crond:session): session closed for user root
>Feb 10 20:05:52 serv2 systemd-journald[28584]: Journal stopped
>Feb 10 20:05:52 serv2 systemd-journald[28725]: System journal (/var/log/journal/) is currently using 1.0G.
>                                               Maximum allowed usage is set to 4.0G.
>                                               Leaving at least 4.0G free (of currently available 32.1G of space).
>                                               Enforced usage limit is thus 4.0G, of which 2.9G are still available.
>Feb 10 20:05:52 serv2 systemd-journald[28584]: Received SIGTERM from PID 1 (systemd).
>Feb 10 20:05:52 serv2 systemd-journald[28725]: Journal started
>Feb 10 20:09:09 serv2 systemd-journald[5269]: System journal (/var/log/journal/) is currently using 1.1G.
>                                              Maximum allowed usage is set to 4.0G.
>                                              Leaving at least 4.0G free (of currently available 32.1G of space).
>                                              Enforced usage limit is thus 4.0G, of which 2.8G are still available.
>Feb 10 20:09:09 serv2 systemd-journald[5269]: Journal started
>Feb 10 20:07:37 serv2 smartd[902]: Device: /dev/sdc [SAT], SMART Usage Attribute: 194 Temperature_Celsius changed from 114 to 113
>Feb 10 20:08:52 serv2 systemd[1]: systemd-journald.service: Watchdog timeout (limit 3min)!
>Feb 10 20:08:01 serv2 clamd[2262]: SelfCheck: Database status OK.
>Feb 10 20:08:27 serv2 freshclam[1491]: Received signal: wake up
>Feb 10 20:08:35 serv2 nscd[882]: nss_ldap: failed to bind to LDAP server ldap://serv2:389: Can't contact LDAP server
>Feb 10 20:08:35 serv2 nscd[882]: nss_ldap: could not search LDAP server - Server is unavailable
>Feb 10 20:09:09 serv2 freshclam[1491]: ClamAV update process started at Fri Feb 10 20:09:09 2017
>Feb 10 20:09:09 serv2 slapd[1755]: connection_input: conn=1021 deferring operation: binding
>Feb 10 20:09:09 serv2 freshclam[1491]: main.cvd is up to date (version: 57, sigs: 4218790, f-level: 60, builder: amishhammer)
>Feb 10 20:09:09 serv2 freshclam[1491]: daily.cld is up to date (version: 23038, sigs: 1538106, f-level: 63, builder: neo)
>Feb 10 20:09:09 serv2 freshclam[1491]: bytecode.cld is up to date (version: 290, sigs: 55, f-level: 63, builder: neo)
>Feb 10 20:09:09 serv2 freshclam[1491]: --------------------------------------
>Feb 10 20:09:10 serv2 systemd-coredump[5264]: Detected coredump of the journal daemon itself, diverted to /var/lib/systemd/coredump/core.systemd-journal.0.ef06c7c4aa3c432bbdfa8e4946720d65.28725.1486753749000000.xz.
>

Actions: View

Attachments on bug 1021307: 711149 | 711994 | 711995 | 713752 | 713777 | 715475 | 715481 | 715482 | 715483 | 715500 | 719297 | 719368