Bugzilla – Bug 237770
beagled is doing too much work
Last modified: 2008-03-07 07:19:46 UTC
Since update from 10.1 to 10.2, I felt the system is less responsive. Also when I was doing nothing, the harddisk made a sound like continuously self-testing. I had also suspected some trojan, but it turned out that beagled is the top CPU process (regarding cumulative CPU): PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 4097 user 21 0 97924 18m 4708 S 7.0 4.8 211:18.46 beagled I agree that I have a lot of files in my homedirectory, but I always had them. It's my computer after all. Other symptoms found while examining the logs: 2007-01-23-12-57-23-IndexHelper: 070123 1257283649 25787 IndexH WARN: Unable to set IO priority for process to idle 2007-01-19-13-31-40-Beagle: 070123 1603259703 04097 Beagle WARN: KMail backend: /home/user/Mail contains a maildir directory but no corresponding index file. Probably not a KMail mail directory. Ignoring this location! (this message repeating over and over; I never used KMail on this machine)
As I'm going to reboot soon, here's the current CPU usage of beagled: PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 4097 user 20 0 97988 14m 3088 S 5.3 3.8 1822:02 beagled System uptime is a little over 7 days now.
It seems that after a reboot beagled has calmed down: Uptime is 1 day 40 minutes, and beagled consumed 2 minutes CPU so far.
(In reply to comment #0) > Other symptoms found while examining the logs: > 2007-01-23-12-57-23-IndexHelper: > 070123 1257283649 25787 IndexH WARN: Unable to set IO priority for process to > idle This is typical. Newer kernels don't allow unprivileged processes to set their IO priority to idle. > 2007-01-19-13-31-40-Beagle: > 070123 1603259703 04097 Beagle WARN: KMail backend: /home/user/Mail contains a > maildir directory but no corresponding index file. Probably not a KMail mail > directory. Ignoring this location! > (this message repeating over and over; I never used KMail on this machine) Roughly how many times is this repeated? Does it go on forever, or just many times and then stop?
The message repeated 1037 times between 2007-01-29 and 2007-01-31, and the file "2007-01-29-08-00-51-Beagle" is the logest one within the Log directory. The file ist still updated with those messages. However the CPU usage of beagled is back to normal at the moment.
I can confirm this bug: beagled runs forever, eating all cpu (actually just 1 cpu) whenever desktop is locked. This computer is turned on about all the time, i lock screen when i leave the university and i unlock it on the next day. so whatever it is doing i'm sure it should have completed it already by now. currently (saturday - i'm accessing by ssh and X is idle there) i have this: top - 15:17:18 up 1 day, 5:27, 29 users, load average: 1.27, 1.21, 1.18 Tasks: 143 total, 5 running, 135 sleeping, 1 stopped, 2 zombie Cpu0 : 54.8%us, 15.3%sy, 0.0%ni, 2.0%id, 26.6%wa, 0.3%hi, 1.0%si, 0.0%st Cpu1 : 3.3%us, 0.7%sy, 0.0%ni, 95.0%id, 1.0%wa, 0.0%hi, 0.0%si, 0.0%st Mem: 2060508k total, 2043868k used, 16640k free, 691892k buffers Swap: 4241152k total, 272k used, 4240880k free, 175120k cached PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 4380 miguel 15 0 1057m 163m 8488 S 70 8.1 812:12.44 beagled 4292 miguel 15 0 264m 141m 21m S 3 7.0 39:20.48 firefox-bin 28458 miguel 15 0 8744 1276 900 R 0 0.1 0:00.04 top 1 root 18 0 804 304 244 S 0 0.0 0:00.87 init I have nothing suspicious at ~/.beagle/Log. All files have the "WARN: Unable to set IO priority for process to idle" thing so i guess the KMail stuff above could be ruled out. Note beagled uses 1GB virtual, is this considered normal behaviour? I'm aware Joe has new rpms at http://repos.opensuse.org/home:/joeshaw/openSUSE_10.2. However i would expect some advice if i should update it right now or try to obtain additional information. Since we have a couple of suse computers like this one, i have interest in having this fixed through normal update channels as well...
one day later, beagled is still running and eating cpu cycles. 4380 miguel 34 19 1057m 163m 8488 S 59 8.1 1678:34 beagled just occurred me to check what files beagled is accessing, which might provide some hint. i will create it as an attachment.
Created attachment 118515 [details] lsof | grep beagled
Total CPU usage on screensaver is the intended behavior (assuming you're not on laptop battery). The idea is that since you're away from your computer, Beagle can index your data as quickly as possible without interfering with your usage of the machine. We should probably make that an option, but that should be filed as a separate enhancement request bug. If the CPU usage extends beyond returning to the machine and turning the screensaver off, that would be a bug. Since a number of these have been fixed in the newer versions, I'd ask that you update first and see if you still see the problem. There have also been substantial fixes to memory usage so the 163 megs RSS should decrease quite a bit (and presumably virtual as well, although that's a harder number to analyze). The default log level as shipped is low and won't give you any useful information about CPU usage or what it's indexing. You can either send the beagled process SIGUSR1 to bump it up, or restart beagled with the --debug option (beagled --replace --debug). That will hopefully help things. Newer versions also allow you to send SIGUSR2 to the beagled-helper process to report what is presently being indexed and how long it's been running. That would only be useful if it were the beagled-helper process that was pegging the CPU, not beagled. The only way to get useful information from the log files would be to run beagled with the --debug option. You can send SIGUSR1 to the running beagled
1) this is a desktop computer (no battery) 2) total CPU usage on screensaver, no CPU usage with screensaver off - so no bug here the problem is about beagled never finishing his work... i will update to the newer version and try again.
(In reply to comment #9) > the problem is about beagled never finishing his work... Ah, ok. It would be good to bump up the debug level and monitor what is going on in the logs. There were issues recently with Beagle continuously recrawling one directory. Those should be fixed with the update. Another thing would be to check to see how many directories you have under your home directory, and if this number is greater than /proc/sys/fs/inotify/max_user_watches. If so, Beagle can't watch all of your files and has to recrawl those directories for changes periodically.
(In reply to comment #10) > Another thing would be to check to see how many directories you have under > your home directory, and if this number is greater than > /proc/sys/fs/inotify/max_user_watches. this is definitely the case: find -type d | wc -l 50941 cat /proc/sys/fs/inotify/max_user_watches 8192
Ok, so that's why Beagle never stops crawling. It shouldn't peg the CPU though. I'm still interested in seeing your logs. Maybe Beagle should ship an init script to bump the max_user_watches value up higher.
At the moment I have 1742 directories with 8192 max watches, and beagled is "tame" (uses little CPU). Could it be a problem for the first boot or log-in only?
(In reply to comment #13) > At the moment I have 1742 directories with 8192 max watches, and beagled is > "tame" (uses little CPU). Could it be a problem for the first boot or log-in > only? Don't think it's just a first-time log-in issue as I can kill beagled-helper and then at some point later it is respawned and consuming my CPU like Mr Creosote. The top of top on my system: PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 10956 ogley 37 12 52200 21m 7616 S 78.1 4.3 334:11.19 beagled-helper Joe, what information do you need, I'll provide what I can. (Info pertinent to my system that's already been provided for other systems follows) ogley@riggwelter:~> rpm -q beagle beagle-0.2.16-4 (current Factory version) ogley@riggwelter:~> find -type d | wc -l 10658 ogley@riggwelter:~> cat /proc/sys/fs/inotify/max_user_watches 8192
James: You're probably seeing a different issue. But in any case, send the beagled-helper process SIGUSR2 and it should print out in the log file what file it's been working on. If it's been a while, it's probably a filter bug. I think 0.2.16 has a problem with SVG files; there's a 0.2.16.1 release I did for SLED... I'm going to put together a 0.2.16.2 to fix the /tmp filling issue in another bug, and I'll submit that for both SLED and 10.3.
confirmed, it seems to be the SVG issue that I'm seeing. 20070222 16:38:15.2064 15113 IndexH WARN: Filtering status (1m29s ago): determining filter for file:///home/install/gnome-themes-extras/gnome-themes-extras.orig/Wasp/icons/scalable/emblems/emblem-mail.svg
James: I just checked in 0.2.16.2 to SLED and STABLE, and checked it into the Beagle OBS project as well. So what's the status of this bug, are the issues fixed, are people still having problems?
A quick spec file change and the OBS version build on Factory and I can confirm that the SVG filter issue seems fixed now.
Some new stats (I have 3205 directories and 26339 files in $HOME, making up 14 GB): Uptime 18 days, beagled-helper using 6.6 days of CPU Uptime 4 days, beagled-helper using 1.15 days of CPU (from "top", up 18 days) PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 4286 wuser 34 19 41272 836 620 S 53.0 0.2 9525:33 beagled-helper (from "ps -ax", up 4 days) PID TTY STAT TIME COMMAND 4121 ? Ssl 0:03 beagle-search --debug /usr/lib/beagle/Search.exe --icon --autostarted 4160 ? Sl 0:13 beagled /usr/lib/beagle/BeagleDaemon.exe --bg --autostarted --indexing-delay 300 4284 ? Sl 1657:10 beagled-helper /usr/lib/beagle/IndexHelper.exe This makes between 1/4th and 1/3rd of the CPU. And it seems as soon as you download a new file, beagle starts to examine it, even while it still grows. Most notabley if an online update is even slower than incredibly slow, it turned out that all the CPU is eaten by beagle, yast or update-status. It proved to be helpful to send a kill -STOP to the beagled-helper while doing an update, surfing the net, or downloading ISO images. After that I send a kill -CONT to beagled-helper. However this cannot be the solution!
Ulrich: Beagle will only index a file when it's been created, or the file was open for writing and then closed. Some apps (Azureus, for example) are broken in this regard and open, write, and close the file for every chunk it receives on download! So some additional information on how you are downloading data would be helpful here. I doubt that the update process influences Beagle's indexing, as only your home directory is indexed by default, and the updater works outside of your home directory. A number of CPU-eating bugs have been fixed over the last year. If this is on an openSUSE 10.2 machine, you might want to upgrade to the latest version, available in the openSUSE build service here: http://download.opensuse.org/repositories/Beagle/openSUSE_10.2/ Lastly, if this is still a problem with the latest versions (0.2.18) on either 10.2 or a 10.3 beta, sending SIGUSR2 to the beagled-helper process will output some useful debugging information in ~/.beagle/Log/current-IndexHelper. If you can attach that to the bug, that'd be great. Unfortunately the version shipped in 10.2 doesn't have this functionality.
Based on Commment #20: - What program is being used to download the files in question which are apparently being indexed piecewise during download? - Are you still seeing these problems with the updated beagle packages from the specified repository? - If so, what gets logged to ~/.beagle/Log/current-IndexHelper if you send SIGUSR2 to beagled-helper?
Being tired of having to kill beagled-helper and update-status to get an usable system, I installed openSUSE 10.3. Things seem better there, so it must have been a bug in 10.2 (nobody wanted to fix).
Closing as nobody can provide requested information.