You are viewing illiterat

James Antill - Explaining: "Warning: RPMDB altered outside of yum."

Jan. 22nd, 2010

08:42 pm - Explaining: "Warning: RPMDB altered outside of yum."

Previous Entry Add to Memories Share Next Entry

What does it mean?

The yum message "Warning: RPMDB altered outside of yum." or, as the yum message said for a few months, "Warning: RPMDB has been altered since the last yum transaction." means some application has altered the rpm database (installed or removed a package) without going through the Yum APIs. This is almost always due to someone using rpm directly (Ie. rpm -ivh blah.rpm), but another possibility is an application built on top of the rpm APIs (Ie. smart, apt, zypp). While it's possible that someone has hacked your machine and altered the rpmdb maliciously, it would have to be done poorly to trigger this warning.

Why has yum started to emit this warning?

There are three main sets of reasoning behind bringing this to the users attention.

New yum features require yum being "the" packaging API

There are now a few features in yum, requested by users of the package management system, that require yum is aware of all package actions on the system. Here a few of the current ones:

  • The most obvious example is "yum history", which records when packages were installed, when and by whom. If yum is not involved in installing/updating/removing/etc. some packages then a lot of the benefits of "yum history" are gone. For instance there is no useful audit trail anymore, you can't use "yum history list blah" and know you have all the instances where something happened to "blah".
  • Yum now has it's own database, for package information it wants to record but has no corresponding entry in the rpmdb, the obvious example is "the id of the repository that this package was installed from" but there are quite a few pieces of info. now.
  • Following on from the previous point, rpmdb versions are a significant feature for managing many machines by yum. They require information from the yumdb, so installing something via. yum on one machine but via. rpm on another would give the machines different "rpmdb versions".
  • This is not a complete list, and as more package management features are implemented they are much more likely to be implemented at the yum layer than at the rpm layer. Not because rpm is bad, but for the same reasons that the above features were implemented in yum, it's much easier and faster to implement them there.

    Rpm is often abused, when used directly

    We, the yum developers, often find that people using rpm directly only do so to solve problems in a way that just creates more problems (and often looks like yum is at fault). For instance using any of --force, --nodeps, --justdb is pretty much outright lying to yum, and is pretty much guaranteed to confuse yum vs. rpm. For instance over the last 3 months, over 10% of the bugs we've had against yum have been due to this kind of problem (and probably very close to 100% of bugs about "rpm doesn't like the depsolving solution yum did").

    There is also the problem of users accidentally using "rpm --install" which is very different from what yum thinks of as an "install" operation. Which, again, just creates problems and even if yum works around these it is unlikely to do so in a way the user wished.

    In addition to the problems it creates there are no known reasons to use rpm instead of yum. Over the last year or so yum has added features like the downgrade and reinstall commands (which obsolete using rpm --oldpackage) and are generally much easier for the user (in this case esp. combined with yum history undo/redo). Recent versions of yum will even allow you to do things like "yum install http://example.com/blah.rpm" which was the last use case where you could easily do something with rpm and not with yum (and even then we also changed yum so we could add the --releasever, which is a much easier solution to the common reason why people want http installs). But if you can think of something that is easier to do with rpm, we'll almost certainly add a feature to yum to solve that problem.

    We do extra work, and so inform the user

    We now do a non-trivial amount of work when we detect that something has happened to the rpmdb, including a full "yum check" run, so that yum can show problems as close to their source as possible. Just stopping yum for a significant amount of time, or outputting problem reports, with no context would not be a good way to interact with the user. We are also able to cache rpm data, and thus. have yum run faster, when yum is aware of all rpmdb operations (so again, we'd want some way to let the user know: yum is running slower, and this is why).

    What can I do to make it stop?

    The most obvious solution is the one we recommend: Only use yum, or tools that use yum APIs, to install/update/remove/etc. packages on your system. If you don't like yum, and prefer to use some other packaging system that's fine but we'd assume you'd want to use that system all the time (and thus. it doesn't matter what yum does or doesn't do, as you won't be using it).

    If you think you really need to use yum some/most of the time but something else the rest of the time, then there is a way to currently turn all this checking off. In your yum.conf set "history_record = false", and yum will not record any history (the "yum history" command will be useless) but yum will also not be able to tell when the rpmdb has changed. This will break other features, like the yumdb, but it will stop the warnings.

    Known problems

    Old versions of the "remove-with-leaves" yum plugin called an API which would corrupt the state yum thought the rpmdb was in. Older versions of the "keys" plugin may also have been affected. This meant that the warning was produced all the time, and yum history would not be happy. Newer versions of yum fixed the APIs affected.

    The akmod packages seem to work by calling rpm directly, and thus. operating without yum's knowledge, they need to be fixed.

    Comments:

    From:https://www.google.com/accounts/o8/id?id=AItOawm11Wk3EgFBySTYh7jnZIc_Yh6afXMl83o
    Date:January 23rd, 2010 11:17 pm (UTC)

    rpm commands i use

    (Link)
    I use the following rpm commands (because i don't of a way to do them in yum, and i haven't checked in a while!)

    1. Find out what package this file belongs to

    rpm -qf /path/to/some/file

    2. And the opposite. Files that belong to this package

    rpm -ql some-package
    [User Picture]
    From:illiterat
    Date:January 24th, 2010 04:18 pm (UTC)

    Query commands are fine to continue to use

    (Link)
    Using any of the query commands/options (anything with --query or -q in the commands line), is fine. Yum only needs to be involved if you are altering the rpm database. The query commands just read information from it.
    [User Picture]
    From:mattdm
    Date:January 24th, 2010 10:42 pm (UTC)

    Re: Query commands are fine to continue to use

    (Link)
    Suggestion: get in the habit of replacing "rpm -q" with "rpmquery". No real difference, but conceptually nicer.
    From:bogado
    Date:January 25th, 2010 12:16 pm (UTC)

    I know a command that rpm does that yum don't.

    (Link)
    or maybe I didn't read the manual very carefully... :-P

    The other day I was trying to install the boxee rpms from web, I couldn't figure out how to make this command using yum :

    # rpm -Ivh http://...

    # yum localinstall -> don't do http I think probably correctly, since the file isn't local.

    # yum install http://... -> ops there is no "http://..." package in the database.

    # yum remoteinstall http://... -> this command don't seem to exist. :)

    I prefer to install official packages, but sometimes there is no official version or maybe I am installing a not-yet-ready for production package that don't have a yum repo associated to it. I like to use the rpm command since it is simple and direct, but it don't resolve dependencies and now you gave even more reasons not to use it, but "wget -> yum localinstall -> rm" is a bit longer then simply "rpm -Ivh http://..."
    [User Picture]
    From:illiterat
    Date:January 25th, 2010 03:48 pm (UTC)

    Re: I know a command that rpm does that yum don't.

    (Link)
    Third paragraph from the "rpm is often abused" section:
    Recent versions of yum will even allow you to do things like "yum install http://example.com/blah.rpm" which was the last use case where you could easily do something with rpm and not with yum (and even then we also changed yum so we could add the --releasever, which is a much easier solution to the common reason why people want http installs)
    ...here "recent" means the 3.2.26 pre-releases, which can be got from the RHEL-6 beta and rawhide.
    From:bogado
    Date:January 26th, 2010 11:40 am (UTC)

    Re: I know a command that rpm does that yum don't.

    (Link)
    Ops. :-) my bad. :-) I didn't read as carefully as I though I have, hehehe.
    From:(Anonymous)
    Date:January 28th, 2010 01:52 pm (UTC)

    Freshen ?

    (Link)
    I am in the habit of downloading koji builds of stuff of interest in /koji (using koji cmdline + buildid) and doing
    rpm -Fhv /koji/*.rpm
    to get whatever is new to get updated.
    I know there are koji repos but using them seems a bit of an overkill.
    I wonder how would I go about doing the -Fhv * part with yum ?
    [User Picture]
    From:illiterat
    Date:January 28th, 2010 02:58 pm (UTC)

    Re: Freshen ?

    (Link)
    There are a couple of ways, first off I'd always recommend having a real koji repo. (that's what I do) then it's the very easy "yum --enablerepo=koji update blah".

    But if you want to continue downloading things manually, you want to install "yum-plugin-tmprepo" and then you can do:

    yum --tmprepo=/path/to/koji/rpms/ update blah

    ...I'm not 100% on if --freshen will follow obsoletes or not, yum will follow it's config. (defaults to yes) and there's no way to turn it off from the cmd. line (although you can turn it off in the config. and turn it on manually).
    From:https://me.yahoo.com/ciupicri#87205
    Date:February 9th, 2010 12:13 am (UTC)

    Re: Freshen ?

    (Link)
    How does your koji repository looks like?
    [User Picture]
    From:illiterat
    Date:February 15th, 2010 03:12 pm (UTC)

    Re: Freshen ?

    (Link)
    The koji-32 repo. isn't needed for non-x86_64 ... but it does no harm.
    [koji]
    name=Koji $basearch
    baseurl=http://koji.fedoraproject.org/static-repos/dist-rawhide-current/$basearch/
    gpgcheck = false
    enabled=false
    
    [koji-32]
    name=Koji i386
    baseurl=http://koji.fedoraproject.org/static-repos/dist-rawhide-current/i386/
    gpgcheck = false
    enabled=false
    
    [User Picture]
    From:abbot
    Date:February 2nd, 2010 11:11 am (UTC)
    (Link)
    I'm usually using `rpm -ivh' / `rpm -Uvh' to install some third-party software like skype or virtualbox. yum startup time for `yum localinstall' is terribly slow compared to simple rpm -ivh / rpm -Uvh. Similar problem with `rpm -e', compared to `yum remove'. It often takes less time to remove some package with `rpm -e' then it takes yum to start.
    From:(Anonymous)
    Date:April 16th, 2010 05:47 am (UTC)

    Yum startup time

    (Link)
    I agree: I'm a regular yum user, but the abysmally slow startup times make me consider alternatives. It appears that repo caching in yum is too pessimistic.

    Is there a daily cron job that will keep yum's cache fresh all the time and allow yum to start in less than 10 seconds, rather than the minute-plus I'm seeing right now?

    Thanks!

    Anonymous -- illiterat.livejournal@ch.pkts.ca
    From:(Anonymous)
    Date:April 21st, 2010 07:34 am (UTC)

    About --nodeps and --force

    (Link)
    Well, I have to say that it happens that I have to use those options. One use case is when I install Zimbra (mail server) that doesn't specify 'smtp' provides and I want to remove exim/postfix for security reasons (and also to not accidentally turn it on). I know that Zimbra packages have to be fixed, but until it is fixed, this is the way I have to do it.

    The other usecase is when I want to remove and then install again existing package. With 'yum remove' it would remove all dependencies also which costs in terms of time, bandwidth, and possibly something could be broken because of that.

    Or, is some a way to achieve this with yum?
    [User Picture]
    From:illiterat
    Date:April 21st, 2010 06:25 pm (UTC)

    Re: About --nodeps and --force

    (Link)

    For the first case, the much better option is to create a dummy package which just requires Zimbra and provides "smtp" (etc.)

    One big problem of doing --nodeps is that when you next install anything new which "requires: smtp" yum will not do it, and even worse if you try and upgrade anything which "requires: smtp" yum will assume it's fine (because the old version required it and is installed) but then rpm will abort the transaction because the --nodeps isn't transative.


    For the second case you want "yum reinstall blah".

    [User Picture]
    From:mrmeval
    Date:May 22nd, 2010 10:08 pm (UTC)
    (Link)
    tl;dr

    How do I fix the problem now that the evil bit has been set?
    [User Picture]
    From:illiterat
    Date:June 3rd, 2010 07:47 pm (UTC)

    How does it "go away"

    (Link)
    It doesn't ever really go away until you do:

    yum history new

    ...however after the first successful transaction, yum will not complain anymore and you can only see that something happened outside of yum by looking at "yum history list".

    Of course if you have something like akmods, which is altering the rpmdb via. rpm on every kernel update ... then you'll have to wait for one more transaction each time it does it.
    [User Picture]
    From:mrmeval
    Date:June 3rd, 2010 09:27 pm (UTC)

    Re: How does it "go away"

    (Link)
    Thank you

    I'd not seen that fix. Since yum and rpm are so integrated they should just merge them. No good anagram comes to mind so here are the bad ones for yumrpm: Mum Pry, My Rump, Mr My Up. So perhaps they should come up with a more creative solution.

    From:(Anonymous)
    Date:December 15th, 2010 01:26 am (UTC)

    Package Kit

    (Link)
    Does this affect anyone who uses packagekit? I've actually been using both packagekit and yum interchangeably in fedora 14. I expect this isn't a problem, since package kit just uses yum as a backend, but who knows... James
    From:(Anonymous)
    Date:July 26th, 2012 04:55 am (UTC)
    (Link)
    Hello.

    Shouldn't rebuilding the rpm database get rid of both the issue with the RPMDB warning and the issue that rises when using --nodeps?
    Just a thought.
    [User Picture]
    From:illiterat
    Date:July 26th, 2012 01:36 pm (UTC)
    (Link)
    No, "rpm --rebuilddb" tries to fix problems with corruption of the rpm DB. When using --force or --nodeps there hasn't been any corruption, the user has just told rpm to do something that goes against the rules (and done so behind yum's back).
    From:(Anonymous)
    Date:June 24th, 2013 10:54 pm (UTC)
    (Link)
    While I understand where you're coming from, as I have written code myself, this is a bit of a deceptive statement:

    "In addition to the problems it creates there are no known reasons to use rpm instead of yum."

    I always attempt to use yum first. However, yum does not always work. It is very 'bubblegum' in my opinion.. I WILL acknowledge the fact that it HAS improved dramatically over the years, however there are several things I can think of that have NOT been addressed yet. Including, off the top of my head, but not limited to:

    - Installing multiple local packages
    - Ignoring dependencies (although you state that this has been resolved in above article, this is not the case
    - Validly reporting history (which is why I was looking this up)

    Why is it that RPM and yum can't get along?
    [User Picture]
    From:illiterat
    Date:June 25th, 2013 02:28 pm (UTC)
    (Link)
    > I always attempt to use yum first. However, yum does not always work. It is very 'bubblegum' in my opinion..

    Bug reports are always useful.

    > - Installing multiple local packages

    I've no idea why this wouldn't work in the normal cases, for weird cases you can use things like --tmprepo to make it much easier for yum.

    > - Ignoring dependencies

    True, yum doesn't do this (I don't believe I ever said it did) ... but that's because it's a terrible idea, the result is still broken even if it appears to work.

    > - Validly reporting history

    I have no idea what this means ... rpm doesn't have any real history, so I'm not sure how using it will help you.

    > Why is it that RPM and yum can't get along?

    They do, just like rpm and db3 get along ... but if you use db3 commands to manually alter the rpmdb behind rpm then you will be told "don't do that" by the rpm developers. It is the same with yum/rpm, for the same reasons (see the first paragraph of this article for a few examples of why).
    From:(Anonymous)
    Date:June 25th, 2013 05:12 pm (UTC)
    (Link)
    >> I always attempt to use yum first. However, yum does not always work. It is very 'bubblegum' in my opinion..
    >
    > Bug reports are always useful.

    Admitted. And I have submitted some, albeit not in quite some time.

    >> - Ignoring dependencies
    >
    > True, yum doesn't do this (I don't believe I ever said it did) ... but that's because it's a terrible idea, the result is still broken even if it appears to work.

    Agreed, but in the real world you can't do much about poorly maintained packages. I avoid this at all costs, however sometimes it's necessary. For instance, I was having an issue the other day attempting to install libevent's development packages. There were several packages which needed to be installed and all depended on one another.. so what happened? I couldn't install one without the other. Using yum did not allow me to install these three at once due to this; rpm did.

    >> - Validly reporting history
    >
    > I have no idea what this means ... rpm doesn't have any real history, so I'm not sure how using it will help you.

    Yes, they do.. I needed to put together an audit history script in order to track who, what and where had lately modified packages. I do have to commend the team on making the recent version of yum MUCH better in this regard, however we are still running some RHEL 4 and 5 systems, for which rpm -qa --last etc. etc. is really the only way to handle it. It's entirely possible that I'm missing something here, and if I am please feel free to correct me so I can rewrite my script. At this point I am using a combination of rpm -qa --last on the older servers and yum history on the newer ones.
    [User Picture]
    From:illiterat
    Date:June 25th, 2013 07:31 pm (UTC)
    (Link)
    > There were several packages which needed to be installed and all depended on one another.. so what happened? I couldn't install one without the other. Using yum did not allow me to install these three at once due to this; rpm did

    Assuming that you didn't use --nodeps with rpm, then yum should have worked fine (they'd all need to be in the same transaction, but that's true of rpm too). It's hard to tell what went wrong here without a real bugreport/information and livejournal isn't the right place anyway. Maybe come by #yum on OpenProjects if you don't want to use bugzilla?

    If you had to use --nodeps, that's different. There are ways to work around it (create virtual packages that provide the missing deps.), and again I would repeat that using --nodeps does not solve any problems, it just hides them (the most common problem being that upgrades can't work).

    > for which rpm -qa --last etc. etc. is really the only way to handle it.

    Ahh, yeh, that's ordering by %{installtime} which is something but far from the amount of info. we have access to now. There's also /var/log/yum.log which contains more data, but yum history was created because that log file was lacking.

    For RHEL-5 we made sure that the RHEL-6.0 yum was buildable/runnable on RHEL-5 and had some plans to rebase it, but although it had a lot of nice features the change was deemed too disruptive. There is also:

    http://james.fedorapeople.org/yum/5-from-6/

    ...which I believe stopped at the 6.0 GA version, if you want to try it. Obviously that can only help if you've already installed it before you need it.

    Also using rpm --qf is 100% supported and we (yum developers) often use it, although repoquery --installed --qf is often easier/friendlier.