Recently I had an issue with my Mac Mini Server and fixing a corrupt Mac OS X Server Open Directory (OD) database.
One of the HDD’s had become unstable and it eventually just died. That left me with one drive available which is ok but it also caused some issues when the RAID array I was running was killed by the drive failure. I always keep backups and regular clones of my drives, but as the drive had been starting to fail for a while there was always a chance of corruption throughout the more recent copies. After decommissioning the dead drive I cloned the most recent copy back to suitable partitions on the secondary drive and booted it up. Everything seemed to be working without issue! Yay!
Then came the most recent OS X update to 10.8.4. Sitting at lunch during work last Friday I decided to go ahead and update the server using Prompt on my iPhone. I SSH’d in as usual and ran the commands for the update. It seemed to update smoothly and then wanted to reboot. After the reboot I reconnected via SSH and checked on the server state. Most of the services were running without issue.
The update did not likely cause any issues by itself, but after the reboot the Open Directory service would not start and upon diving a bit more deeply into the terminal I found that it was due to corruption. I tried some fixes and some restarts but to no avail. I was feeling rather gloomy that I might have to go back to a significantly older version on my server to get the OD working again even though everything else seemed fine.
Lunch was over, and there did not seem like much to do with the issue until I got home so I went back to work and tried to forget about it (unsuccessfully). Once work was finished I hopped on the subway and started researching the issue. I found many various fixes but most of the did not seem appropriate or did not work when I tried them. After a bit of research I started to see several sites mention certain slapd related fixes that sounded promising, so I fired up Prompt on my iPhone and logged in.
First of all, I used launchctl to unload the openldap:
$ sudo launchctl unload /System/Library/LaunchDaemons/org.openldap.slapd.plist
…then I ran a recommended recovery:
$ sudo db_recover -h /var/db/openldap/authdata/
To see how things went, I ran slapd with tool mode switches:
$ sudo /usr/libexec/slapd -Tt
…and it gave me this response:
53f31f93 bdb_monitor_db_open: monitoring disabled; configure monitor database to enable
config file testing succeeded
That looks promising, so I turned Open Directory back on:
$ sudo launchctl load /System/Library/LaunchDaemons/org.openldap.slapd.plist
The OD seemed to be working fine, but I wanted to reboot just to make sure it was stable enough:
$ sudo shutdown -r now
After the reboot everything (including OD) came up fine! I realized that people on the train were probably wondering why I was looking so excited peering at a black screen on my phone, but I still enjoyed my small victory 😉
THANKS A MILLION!!! Just upgraded to Server 4.1.3 and had lost all users and groups :0 Thanks to your instruction, they were all recovered. Spent more time looking for the solution than applying fix. Can’t thank you enough!!!
I’m glad that it helped! It is getting a bit old now, but I’m glad to hear that it still works on newer versions of Server. I (luckily) have not had occasion to test it recently.
There’s not much on this subject, but every time I seem to try to update the OS (probably, more likely, when the system reboots), I lose my open directory. I’m wondering if this has been tested or if you know those same commands would work in Yosemite. I would like to repair the DB before it crashes again and before I upgrade to El Capitan and then Sierra this fall. Any help would be appreciated.
Thanks a lot. You saved the day.I do not know why but I lost my users.Now they are back.