Bug 666450

Summary: smbd crash on start, cannot opnen secrets.tdb
Product: [openSUSE] openSUSE 11.4 Reporter: Joachim Reichelt <Joachim.Reichelt>
Component: SambaAssignee: Jeff Mahoney <jeffm>
Status: RESOLVED FIXED QA Contact: The 'Opening Windows to a Wider World' guys <samba-maintainers>
Severity: Major    
Priority: P5 - None CC: chrysantine, chucktr, davide.verne, doerges, forgotten_RGNLqzyWVb, forgotten_taWGjDL4xO, harbrink, joachim.banzhaf, lauffer, magendiran, maintenance, martin.schlander, meissner, plinnell, samba-maintainers, silviu_marin-caea, squealernet, suse-beta
Version: Factory   
Target Milestone: ---   
Hardware: x86-64   
OS: Other   
Whiteboard: maint:released:11.4:41905
Found By: --- Services Priority:
Business Priority: Blocker: ---
Marketing QA Status: --- IT Deployment: ---
Deadline: 2011-07-25   
Attachments: strace -f `which smbd` > 1 2&>a
bzip2 audit.log ...
/etc/apparmor.d/usr.sbin.*mbd as tar
/etc/apparmor.d/usr.sbin.nmbd

Description Joachim Reichelt 2011-01-23 20:31:41 UTC
Created attachment 409705 [details]
strace -f `which smbd` > 1 2&>a

User-Agent:       Mozilla/5.0 (X11; Linux x86_64; rv:2.0b9) Gecko/20110110 Firefox/4.0b9

I cannot start smbd. It immideately crashes silently.

Reproducible: Always

Steps to Reproduce:
rcsmbd start
ps -ef | grep smbd (is empty)



To see what is going on I did an strace:
strace -F /usr/sbin/smbd
after:
rpm -e samba ... (all pakages with samba in the name
rm -rf /etc/samba /var/lib/sambe /var/log/samba
zypper in samba-client samba
strace -f `which smbd` > 1 2&>a

File a is attached
Comment 1 Joachim Reichelt 2011-01-23 20:57:54 UTC
*** Bug 666451 has been marked as a duplicate of this bug. ***
Comment 2 Lars Müller 2011-01-24 17:19:13 UTC
Are you using apparmor?

If yes, please disable it and try again to ensure your (or the current default) apparmor configuration doesn't cause this.
Comment 3 Joachim Reichelt 2011-01-25 18:17:10 UTC
It seems to be appamor.

# rcappamor stop
# rcsmb start
# ps -ef | grep mbd
root      3487     1  0 19:11 ?        00:00:00 /usr/sbin/nmbd -D -s /etc/samba/smb.conf
root      4401     1  0 19:11 ?        00:00:00 /usr/sbin/smbd -D -s /etc/samba/smb.conf
root      4403  4401  0 19:11 ?        00:00:00 /usr/sbin/smbd -D -s /etc/samba/smb.conf
root      5614  5517  0 19:15 pts/0    00:00:00 grep mbd

======

I did not change anything in appamor after upgrade to 11.4m*
So this is the default (or some leftover)
Comment 4 Joachim Reichelt 2011-01-25 18:48:55 UTC
O.K.
I checked the appamor install.
There are TWO trees under /etc:
/etc/appamor
/etc/appamor.d


/etc# grep -r secrets. apparmor*
apparmor/profiles/extras/usr.sbin.smbd:  /etc/samba/secrets.tdb rw,
apparmor/severity.db:/etc/ppp/*secrets  8 6 0

So I added the /etc-lines from apparmor/profiles... to the
apparmor.d/... file and started apparmor.
Now smbd stops with:

[2011/01/25 19:40:35.627021,  1] lib/util_tdb.c:521(tdb_wrap_log)
  tdb(unnamed): tdb_open_ex: failed to get global lock on /etc/samba/secrets.tdb: Keine Berechtigung
[2011/01/25 19:40:35.627267,  0] passdb/secrets.c:73(secrets_init)
  Failed to open /etc/samba/secrets.tdb
[2011/01/25 19:40:35.627618,  0] smbd/server.c:1235(main)

So it is a misconfiguration off apparmor.
Comment 5 Lars Müller 2011-01-26 10:05:46 UTC
IIRC this is a known apparmor issue.  But Jeff will know this for sure.
Comment 6 Jeff Mahoney 2011-01-26 14:24:29 UTC
Please attach your /var/log/audit/audit.log.
Comment 7 Joachim Reichelt 2011-01-26 22:38:20 UTC
Created attachment 410615 [details]
bzip2 audit.log ...

Full log.
This problem is related to:
# uname -a
Linux Joachim-PC 2.6.37-20-desktop #1 SMP PREEMPT 2011-01-22 00:41:44 +0100 x86_64 x86_64 x86_64 GNU/Linux

Look at the lines starting about line 1500.
Comment 8 Jeff Mahoney 2011-01-26 23:01:02 UTC
Ok. Sure looks like an apparmor bug. Can you make sure you have the latest security:apparmor:factory apparmor package set[1] installed and use logprof to add the missing components to the profile? It may take a few cycles of starting smbd, having it fail, and running logprof again.

Then post your versions of /etc/apparmor.d/usr.sbin.[sn]mbd

[1] http://download.opensuse.org/repositories/security:/apparmor:/factory/openSUSE_Factory/
Comment 9 Joachim Reichelt 2011-01-27 21:02:13 UTC
There was a minor problem with nmbd:
The right to access mode w to /var/log/samba/cores/ was missing.

But:
As smbd did not start at all, I added to /etc/apparmor.d/usr.sbin/smbd
one line:
 /etc/samba/secrets.tdb rwk,

Now smbd is up, but:
/var/log/samba/log.smbd:
# tail -20 /var/log/samba/log.smbd 
  tdb(unnamed): tdb_open_ex: failed to get global lock on /etc/samba/secrets.tdb: Keine Berechtigung
[2011/01/27 21:30:10.645532,  0] passdb/secrets.c:73(secrets_init)
  Failed to open /etc/samba/secrets.tdb
[2011/01/27 21:30:10.645638,  0] smbd/server.c:1235(main)
  ERROR: smbd can not open secrets.tdb
[2011/01/27 21:31:51,  0] lib/fault.c:250(dump_core_setup)
  Unable to setup corepath for smbd: Permission denied
[2011/01/27 21:31:51,  0] smbd/server.c:1135(main)
  smbd version 3.5.6-2486-SUSE-SL11.4-x86_64 started.
  Copyright Andrew Tridgell and the Samba Team 1992-2010
[2011/01/27 21:31:51.714608,  0] passdb/pdb_tdb.c:420(tdbsam_open)
  tdbsam_open: Failed to open/create TDB passwd [/etc/samba/passdb.tdb]
[2011/01/27 21:31:51.714710,  0] passdb/pdb_tdb.c:549(tdbsam_getsampwnam)
  tdbsam_getsampwnam: failed to open /etc/samba/passdb.tdb!
[2011/01/27 21:31:51.721156,  0] smbd/server.c:500(smbd_open_one_socket)
  smbd_open_once_socket: open_socket_in: Die Adresse wird bereits verwendet
[2011/01/27 21:31:51.721317,  0] smbd/server.c:500(smbd_open_one_socket)
  smbd_open_once_socket: open_socket_in: Die Adresse wird bereits verwendet
[2011/01/27 21:34:51.908167,  0] smbd/server.c:281(remove_child_pid)
  Could not find child 7084 -- ignoring

==
/etc/apparmor.d/usr/sbin/nmbd read now:


# Last Modified: Thu Jan 27 21:27:07 2011
#include <tunables/global>

/usr/sbin/nmbd {
  #include <abstractions/base>
  #include <abstractions/nameservice>
  #include <abstractions/samba>

  capability net_bind_service,



  /usr/sbin/nmbd mr,
  /var/cache/samba/browse.dat* rw,
  /var/lib/samba/wins.dat* rw,
  /var/log/samba/cores/ w,
  /var/log/samba/cores/nmbd/ rw,
  /var/log/samba/cores/nmbd/** rw,
  /var/run/samba/** rk,
  /var/run/samba/nmbd.pid rw,

}

This is apparmor as in openSUSE 11.4-factory just now.

Now trying from security:...
 zypper se -si apparmor
Daten des Repositorys laden ...
Installierte Pakete lesen ...

S | Name                           | Typ   | Version          | Arch   | Repository       
--+--------------------------------+-------+------------------+--------+------------------
i | apparmor-docs                  | Paket | 2.5.1-45.1       | x86_64 | (Systempakete)   
i | apparmor-parser                | Paket | 2.5.1.r1445-64.1 | x86_64 | Security-Apparmor
i | apparmor-profiles              | Paket | 2.5.1.r1445-64.1 | x86_64 | Security-Apparmor
i | apparmor-utils                 | Paket | 2.5.1.r1445-64.1 | noarch | Security-Apparmor
i | libapparmor1                   | Paket | 2.5.1.r1445-64.1 | x86_64 | Security-Apparmor
i | libapparmor1-32bit             | Paket | 2.5.1-45.1       | x86_64 | (Systempakete)   
i | pam_apparmor-32bit             | Paket | 2.5.1-45.1       | x86_64 | (Systempakete)   
i | patterns-openSUSE-apparmor_opt | Paket | 11.3-42.1        | x86_64 | Factory-OSS      
i | perl-apparmor                  | Paket | 2.5.1.r1445-64.1 | x86_64 | Security-Apparmor
i | yast2-apparmor                 | Paket | 2.20.0-1.2       | noarch | Factory-OSS      

but the problem is the same. nmbd is fixed the same way as before, smbd cannot
read/write /etc/samba/passdb.tdb
Comment 10 Jeff Mahoney 2011-01-27 21:10:01 UTC
Well, yeah. What you're running into is what I meant by "It may take a few cycles of starting smbd, having it fail, and running logprof again."

Another way to do it is to put the profile in complain mode by adding the flag to the profile like this:

/usr/sbin/smbd (flags=complain) {
...
}

.. that will essentially run smbd unprotected but will still generate the events so you can update the profile.
Comment 11 Chuck Taylor 2011-02-06 00:22:03 UTC
Well, let me throw in a monkey wrench. I do NOT have apparmor running at all and I still can not get smb or nmb to run. The complaint is ... cannot read the files in /etc/samba. What has changed on the permissions since 11.3 and are we now making it a requirement to run apparmor???

By the way, what is this secrets.tbd file. There is not one on my system.
Comment 12 Chuck Taylor 2011-02-06 03:01:01 UTC
This is an Update.  I was wrong. I did have apparmor running. I had tried to check it with a ps-ef | grep apparmor and also with the rcapparmor command. Neither worked so I thought it wasn't running. After checking System Services Runlevels I found that aaeventd and boot.apparmor wee running. I stopped them both and was able to finally get smb and nmb running.
Comment 13 Joachim Reichelt 2011-02-20 20:29:50 UTC
Created attachment 415186 [details]
/etc/apparmor.d/usr.sbin.*mbd as tar

Working files for samba:
/etc/apparmor.d/usr.sbin.smbd
/etc/apparmor.d/usr.sbin.nmbd
Comment 14 Bin Li 2011-03-02 05:49:23 UTC
(In reply to comment #13)
> Created an attachment (id=415186) [details]
> /etc/apparmor.d/usr.sbin.*mbd as tar
> 
> Working files for samba:
> /etc/apparmor.d/usr.sbin.smbd
> /etc/apparmor.d/usr.sbin.nmbd

With this file the rcsmb start successfully, but the rcnmb start failed.

Mar  2 13:47:50 ATong nmbd[17259]: [2011/03/02 13:47:50.869205,  0] nmbd/nmbd.c:861(main)
Mar  2 13:47:50 ATong nmbd[17259]:   error opening config file
Comment 15 Joachim Reichelt 2011-03-08 21:37:18 UTC
Created attachment 418197 [details]
/etc/apparmor.d/usr.sbin.nmbd 

There is one line to change!
/var/lib/samba/browse.dat. rw,

I had only "w" as the file was empty that time.
Comment 16 Eberhard Harbrink 2011-03-12 19:56:44 UTC
I don't see it mentioned above, but for me it was also necessary to insert

 /etc/samba/passdb.tdb rwk,

into /etc/apparmor.d/usr.sbin.smbd .
Otherwise I would see the server when browsing, but I couldn't log in.
Comment 17 Jeff Mahoney 2011-03-14 19:00:48 UTC
I've added the files and dirs to the profiles. Test packages should appear at http://download.opensuse.org/repositories/home:/jeff_mahoney:/branches:/openSUSE:/11.4:/Update:/Test/standard shortly.

Please test and report back.
Comment 18 Eberhard Harbrink 2011-03-14 19:33:35 UTC
Seems to work for me now.
Comment 19 James McDonough 2011-03-15 15:34:26 UTC
*** Bug 679501 has been marked as a duplicate of this bug. ***
Comment 20 Davide Vernè 2011-03-15 22:30:58 UTC
It works for me, too

2.6.37.1-1.2-desktop #1 SMP PREEMPT 2011-02-21 10:34:10 +0100 (i586)
Comment 21 Marcus Meissner 2011-03-16 12:25:11 UTC
needinfo maintenance@opensuse.org for an update when done
Comment 22 Heidi Lahtinen 2011-03-17 19:00:48 UTC
I upgraded an old server and tested with Jeff's packages (latest) and ran into at least one file that is not covered by the AppArmor profile there and will cause issues issues;

[2011/03/17 20:48:32.485909,  1] lib/server_mutex.c:64(grab_named_mutex)
  Could not open mutex.tdb: Permission denied

/var/lib/samba/mutex.tdb
Comment 23 Marcus Meissner 2011-03-17 20:35:49 UTC
you can run "logprof" to check for apparmor denied events and allow them
or the YAST AppArmor -> Update ? Prfoiles? wizard
Comment 24 Forgotten User nWM7y1foa6 2011-03-18 12:23:07 UTC
Interestingly enough, my apparmor already was disabled according to the runlevel services (and chkconfig), but apparmor was really running in the background.  Thanks for the logprof command as it confirmed to me that smb was definitely being blocked.  It didn't work when I "Allowed" the service, but at least I found that apparmor was running (ps -ef never showed any apparmor processes either).

rcapparmor stop did the trick.....   Now I just need to find out why it is loading in the first place, when I have it disabled . . . .
Comment 25 Jeff Mahoney 2011-03-21 15:24:34 UTC
(In reply to comment #22)
> I upgraded an old server and tested with Jeff's packages (latest) and ran into
> at least one file that is not covered by the AppArmor profile there and will
> cause issues issues;
> 
> [2011/03/17 20:48:32.485909,  1] lib/server_mutex.c:64(grab_named_mutex)
>   Could not open mutex.tdb: Permission denied
> 
> /var/lib/samba/mutex.tdb

Can you attach your /var/log/audit/audit.log?
Comment 26 Heidi Lahtinen 2011-03-23 10:28:20 UTC
(In reply to comment #25)
> Can you attach your /var/log/audit/audit.log?

Sorry Jeff, we ran into other issues on the server (not related to the upgrade or AppArmor) and lazed the entire system, including logs.

However I did not run into any other files other than that mutex.tdb that it complained about.
Comment 27 P Linnell 2011-04-01 19:45:41 UTC
I installed the rpms from Jeff's repos and then switched apparmor to enforce mode, then restarted all the samba daemons and all is well. 

However, when I tried to start the event logger I get this error:


rcaaeventd start
Starting AppArmor Event daemon                                                                                                                                           done
1server:/home # Can't locate File/Tail.pm in @INC (@INC contains: /usr/lib/perl5/site_perl/5.12.3/i586-linux-thread-multi /usr/lib/perl5/site_perl/5.12.3 /usr/lib/perl5/vendor_perl/5.12.3/i586-linux-thread-multi /usr/lib/perl5/vendor_perl/5.12.3 /usr/lib/perl5/5.12.3/i586-linux-thread-multi /usr/lib/perl5/5.12.3 .) at /usr/sbin/aa-eventd line 33.
BEGIN failed--compilation aborted at /usr/sbin/aa-eventd line 33.


zypper in perl-File-Tail added the missing module and now aaeventd starts, samba is working and apparmor seems to be working fine in enforce mode.

So, there is a missing Build:Requires or Requires somewhere. After reporting this, I will look in OBS and if possible send an SR for the fix. 

That said, I think this should be a priority maintenance fix. Samba not working out of the box on a default install is not good.

Let me know if you need more info or testing.
Comment 28 Jeff Mahoney 2011-04-11 19:56:28 UTC
I've updated the profile to allow /var/lib/samba/** rwk.
I've updated apparmor-utils to depend on perl-File-Tail.

SR 66522

Test packages again at:

http://download.opensuse.org/repositories/home:/jeff_mahoney:/branches:/openSUSE:/11.4:/Update:/Test/standard/
Comment 29 Heidi Lahtinen 2011-04-15 09:25:44 UTC
(In reply to comment #28)
> Test packages again at:

Works like coffee in the morning - push 'em out?
Comment 30 Forgotten User RGNLqzyWVb 2011-04-16 04:34:27 UTC
I updated apparmor from http://download.opensuse.org/repositories/home:/jeff_mahoney:/branches:/openSUSE:/11.4:/Update:/Test/standard

smbd and nmbd are started, but smbd cannot access the shared dir:

type=AVC msg=audit(1302928001.423:3198): apparmor="DENIED" operation="open" parent=2686 profile="/usr/sbin/smbd" name="/mnt/d04/pub/" pid=10299 comm="smbd" requested_mask="r" denied_mask="r" fsuid=65534 ouid=0
Comment 31 Christian Boltz 2011-04-17 11:52:27 UTC
(Resetting needinfo to Joachim - IMHO there was enough feedback from other people. Joachim, you may still add your comment of course ;-)

(In reply to comment #30)
> ...parent=2686 profile="/usr/sbin/smbd" name="/mnt/d04/pub/" pid=10299 ...

You are opening a can of worms ;-) because samba shares can basically be every directory on your system depending on the samba config.

The profile has
  @{HOMEDIRS}/** lrwk,
which means read and write permissions for home directories (/home/*).

There are two options to solve this in a clean way:
a) edit /etc/apparmor.d/tunables/home or (better)
   /etc/apparmor.d/tunables/home.d/site.local and add your /mnt/d04/pub 
   directory to @{HOMEDIRS}
b) have a separate tunable for samba shares, maybe 
   /etc/apparmor.d/tunables/samba. It could contain:
       @{SMBSHARE}=@{HOMEDIRS} /mnt/d04/pub
   (default value should be @{HOMEDIRS})

Jeff, what do you think about having a separate @{SMBSHARE} tunable?
Comment 32 P Linnell 2011-04-17 15:13:10 UTC
The profile issues perhaps handled in a different bug, but I would ask that this get pushed out now as a maintenance fix. Obviously more than a few folks are affected by this.
Comment 33 Jeff Mahoney 2011-04-17 15:39:59 UTC
Ah. I didn't update the status on this one. The package has already been pushed to the update process.

@cboltz: Yeah, that's definitely the right idea, and what AppArmor 2.6 already does. I wouldn't be opposed to adding tunable profiles like that, so long as they match what's upstream already.

Dmitri, can you open a separate report for that? This one should be closed as the original issue has been fixed.
Comment 34 Forgotten User RGNLqzyWVb 2011-04-18 00:04:49 UTC
(In reply to comment #33)

I created new report, bug #688040.
https://bugzilla.novell.com/show_bug.cgi?id=688040
Comment 35 Bernhard Wiedemann 2011-04-28 11:51:46 UTC
This is an autogenerated message for OBS integration:
This bug (666450) was mentioned in
https://build.opensuse.org/request/show/66464
https://build.opensuse.org/request/show/66522
Comment 36 David Disseldorp 2011-05-15 16:05:23 UTC
*** Bug 693900 has been marked as a duplicate of this bug. ***
Comment 37 Bernhard Wiedemann 2011-06-23 19:00:22 UTC
This is an autogenerated message for OBS integration:
This bug (666450) was mentioned in
https://build.opensuse.org/request/show/74415 11.4 / apparmor
Comment 38 Bernhard Wiedemann 2011-06-24 15:00:20 UTC
This is an autogenerated message for OBS integration:
This bug (666450) was mentioned in
https://build.opensuse.org/request/show/74457 11.4 / apparmor
Comment 39 Swamp Workflow Management 2011-06-25 19:57:59 UTC
The SWAMPID for this issue is 41833.
This issue was rated as low.
Please submit fixed packages until 2011-07-25.
Also create a patchinfo file using this link:
https://swamp.suse.de/webswamp/wf/41833
Comment 40 Swamp Workflow Management 2011-07-07 13:17:25 UTC
Update released for: apache2-mod_apparmor, apparmor-docs, apparmor-parser, apparmor-profiles, apparmor-utils, libapparmor-devel, libapparmor1, pam_apparmor, perl-apparmor, tomcat_apparmor
Products:
openSUSE 11.4 (debug, i586, x86_64)