Following the successful migration of the rest of the infrastructure, it was due time for me to overhaul the web server a bit as well. This was a necessity both because Debian Lenny support was dropped in February, as well as due to cruft that got accumulated from the previous mail server set-up.
Like the new server, the old server was running an Apache web server as well. At some point I found out about mod_ruid and decided to deploy it on Apache. This allowed me to execute the processes running PHP code under privileges of user/group owning the file, using the Linux capabilities.
In the upgrade process I've went for the mod_ruid2 available from the SourceForge. This time around I decided to build the module externally on a dedicated build machine, and then copy it over to my production web server. While I was at it, I also decided to make my first Debian package. This proved to be a bit of hit-and-try effort, though. The Debian tutorials out there work great with standardised stuff like GNU autobuild tools, or cmake, but since mod_ruid2 is compiled by a single command (not even a Makefile is available), it's a bit more tedious process to properly create a package. One day of messing with it all, and I did manage to build it, so all was well (in the process now I at least have a dedicated build machine for Debian Squeeze).
On the original server the schema I went for was having the PHP applications executed as the same user which logs in onto the server to administer them. This does introduce a bit of insecurity (theoretically a malicious or vulnerable script could access contents of user's home, including stuff like authorized_keys etc).
On the new server I've set out to remove this kind of potential attack vector by introducing a separate user which will be running the processing executing the PHP scripts. I also wanted to maintain the ability of regular users to administer the files (so I wouldn't need root when working with the files). Part of this requirement was because of another website which I'm not maintaining myself is hosted on my server. The other part was to remove the possibility of malicious interaction between multiple PHP applications.
My first thought, since I've worked with those a little bit previously, was using the POSIX ACL. To cut the story short, it led to a lot of frustration. There are several major design flaws that render POSIX ACL's virtually useless. You set-up the necessary rules for the document root directory, including the default rules for assigning ACL's to children created within it, but then you get hit by issues like mask being overridden from default values because the originating directory (which you are copying into the document root) has more restrictive permissions (say 700). And then, the final killer was the fact that Apache processes (or PHP?) creating the directories completely ignored the POSIX ACL (they wouldn't add any kind of ACL to children). This was true also for some of the PHP scripts which were checking read/write permissions (they wouldn't let me write to a directory although ACL set in place allowed the user/group to do so).
Needless to say, I finally gave up on POSIX ACL, and came-up with another solution. For each web application I've created a separate user/group. Then I added my user to this dedicated group. After that I set-up the document root directory to be owned by my own user, and the web user's group. The directory also received the permissions of 750, and on top of that it had its sticky bit set for the group. I also made sure my user's umask was set to 0027, so for most files it will work out fine (i.e. I don't want the web user to be able to overwrite PHP scripts themselves, and I can explicitly allow it to write to a limited set of directories).
With that in place, what was left was to configure the mod_ruid2. mod_ruid2 is capable of using either stat-based or config-based set-up. Stat-based means that it will take the ownership of a file (i.e. calling the ''stat()'' function), and assign that user/group to the process running the script. Config-based means that you can explicitly set the user/group which will be running the process executing the script within some context (vhost/directory etc). I've picked the config-based approach so I could have full control (to be more precise, globally I enabled stat-mode, and then I overrode this for each virtual host where I deemed it necessary).
mod_ruid2 also allows you to set the minimum user/group ID which is allowed to run the process running the script. This way you can prevent it from running a PHP script as ''root.root'' just because the ownership over a file was set-up like that, and force some predefined user/group to run the process instead. There's a small catch to this, though.
Let's say you set-up for some directory to be served under the ''www-data.www-data'' user/group (static files, and you don't want to add more web users - yes, mod_ruid2 also serves the files using the same principle of switching user/group for the process), have mod_ruid2 switch user/group to file's only with uid/gid greater than 2000, and have it default to ''www-data.www-data'' if uid/gid is below that value. Now, although it might seem that this should work, it actually doesn't. I've done this on one of the directories from which I serve various files directly, and Apache had always presented me with a permission denied page. I've worked around this by having the directory group sticky bit set, mode to 750, and ownership being such that the group is www-data and user is my main account (so when I copy the files over, they will be accessible to the www-data group). I've also removed the explicit user/group setting from the definition for that directory, and I was able to proceed further. One good thing is that all of my web users start from uid/gid 2000, and regular accounts are starting from 1000 (and not exceeding the limit of 2000). This way, with mod_ruid2's minimum uid/gid set to 2000, the following happens:
<ol> <li>User requests access to a file from the directory</li> <li>Apache hooks with mod_ruid2</li> <li>mod_ruid2 figures out that the uid/gid of file owner are too low</li> <li>mod_ruid2 switches the process to default user/group specified in its configuration</li> <li>The process is able to serve the file without problems due to permissions set beforehand</li> </ol>
All in all, currently I'm very satisfied with the solution I deployed. The only annoyance is that the permissions can be broken if using the ''mv'' command instead of ''cp'', so that's something to pay attention to when you are using a set-up like this (sticky bit won't get propagated with ''mv''). Still, at least it works.
As for the final conclusion, I really wish that someone picks-up the POSIX ACL standard once again, and tries to iron it out so that it becomes usable in real world. There are some things you can't solve with the traditional *nix permissions/ownership system, primarily the problem when you want to add permissions to write/read/whatever on some directory which is already writeable/readable by some other group (you usually end-up making "container" directories or additional groups, and both of these tend to get messy at some point). Additionally, with the quirks ironed-out, POSIX ACL would be a _great_ tool for stuff like Samba even in larger organisations (you'd define a single share, and let the file-system take care of access permission completely). Another afterthought is that move operations can be a real bitch to overcome when you want to inherit parent group etc (sticky bit). It'd be really nice if you could somehow mount a filesystem and tell it to not preserve the permissions when moving files around.
Harum on #
Hi Branco, thanks for the writeup, especially about the POSIX ACL.
I am also evaluating either mod_ruid2 or Apache MPM ITK. Both available on Squeeze. I currently gravitate towards ITK because it allows FastCGI, mod_php, even mod_perl to be contained in a separate user/group. Have you tried/played with ITK?
Branko Majic on #
Glad that you found it useful. I think POSIX ACL is probably best to steer clear from (unless something changes in the standard that'd make it more useful).
I haven't tested MPM ITK, to be honest. I went for the mod_ruid (and then mod_ruid2) myself based on recommendation from one of the sysadmins I know. Oh, and with mod_ruid2 you can have separate user for anything you serve out there - it's not limited to only PHP. You can even serve static files under different user/group. If you happen to do any kind of performance testing on these two, please let me know, it would be a useful thing to know :)
Oh, are you sure that mod_ruid2 is available under Squeeze? I've seen it has landed into Wheezy, but looking at Squeeze repos I can't find it (I built the package myself by hand).
For the record, in the meantime I've realised that many other script modules out there support user/group-switching natively (like mod_passenger or mod_wsgi). mod_php5 is, well, "special" it seems :)
Harum on #
I did some testing today. libapache2-mod-ruid2 is indeed not on Squeeze, sorry my bad.
MPM ITK also allows us to serve anything under a certain user/group.
Here are some of the differences I found:
* With MPM ITK, the Apache child processes are running as root. With mod_ruid2, they are running as www-data. When serving requests, both will switch to specified user/group though.
* When executing CGI: With MPM ITK, Apache child processes first fork, setuid, then create process once more to execute the CGI script (double fork). With mod_ruid2, the Apache parent process setuid the child to user/group, then the child create process to execute the CGI script (only one new process is created). When serving static pages, no forking is done with both modules.
* mod_ruid2 is only available on Linux, I think. While MPM ITK should also be available under FreeBSD (which Debian also runs).
* mod_ruid2's performance for CGI/PHP is slightly better. mod_ruid2's performance for static files is much better, as with MPM ITK static serving suffers a dramatic performance hit. Here are some numbers done with 'ab -n 1000 -c 5' on my PC:
mpm=prefork, static file: 19241 requests/s
mpm=prefork, CGI: 1827 requests/s
mpm=prefork, PHP (mod_php): 15509 requests/s
mpm=itk, static file: 2565 requests/s (7.5x slowdown compared to prefork!)
mpm=itk, CGI: 1082 requests/s
mpm=itk, PHP (mod_php): 1809 requests/s
mpm=prefork, mod_ruid2, static file: 14975 requests/s
mpm=prefork, mod_ruid2, CGI: 1869 requests/s
mpm=prefork, mod_ruid2, PHP (mod_php): 12616 requests/s
* mod_ruid2 allows specifying extra groups. Not sure if that's useful in most cases, but some might need it.
I think I'll go with mod_ruid2 for now. Hope that's useful.
BTW, the captcha is very very hard to see. Took me multiple retries. I almost gave up.
Branko Majic on #
Great, thanks a lot for this performance testing! Seems to be a really huge difference. I'm aware of the Linux-only thing. Unless FreeBSD started supporting the same POSIX.1e extension as well in the meantime (since this extension never left draft, I think POSIX ACL Is part of it as well)?
I'll have to maybe start using two-factor captcha. I had to increase the captcha difficulty to prevent bots from solving it, but apparently it's too annoying, so I'll have to make it a bit simple and see how many bots manage to solve it.