Improve aliases of portage

[blackteahamburger]

Removed 'ers', improved 'emfu', added 'ec'.

[akinomyoga]

Thank you for your contribution.

Removed 'ers',

I'm not familiar with emerge, but as far as I see the man page found online, er appears to be the one that should be modified. I.e., er should be emerge --clean and ers should be emerge --depclean. The command emerge -c seems to be the synonym of emerge --depclean, so er seems wrong while ers seems correct.

improved 'emfu',

Could you explain the background? What would be the flags -Uj and -N? As far as I read the man pages -U seems cleverer than -N but maybe that causes an issue? Also, specifying -j seems faster. Anyway, I'm not familiar with it. Could you explain those changes?

[blackteahamburger]

Thank you for your contribution.

Removed 'ers',

I'm not familiar with emerge, but as far as I see the man page found online, er appears to be the one that should be modified. I.e., er should be emerge --clean and ers should be emerge --depclean. The command emerge -c seems to be the synonym of emerge --depclean, so er seems wrong while ers seems correct.

--clean
Cleans up the system by examining the installed packages and removing older packages. This is accomplished by looking at each installed package and separating the installed versions by slot. Clean will remove all but the most recently installed version in each slot. Clean should not remove unslotted packages. Note: Most recently installed means most recent, not highest version.

--clean only removes old slots so its functionality is limited, I don't believe it is widely used.

improved 'emfu',

Could you explain the background? What would be the flags -Uj and -N? As far as I read the man pages -U seems cleverer than -N but maybe that causes an issue? Also, specifying -j seems faster. Anyway, I'm not familiar with it. Could you explain those changes?

Gentoo wiki says:

Do not use --changed-use(-U) when updating packages on the binary package host, doing so will cause packages with added or removed USE flags to be skipped, which will cause their installation from binary package on the client to fail due to non-matching USE between the source ebuild and binary package (if the client's --binpkg-respect-use=y, the default). Use --newuse(-N), which will always rebuild packages even for added or removed USE flags, ensuring the binary package stays in sync with the source ebuild.

With this in mind I replaced -U with -N.

Also, I think -j should be added by the user themselves because emfu should focus on completing its own functions.

[akinomyoga]

Thank you for your clarifications!

I first need to clarify that my requests are related to backward compatibility but not to the ideal design. If this PR would be the one that introduces new aliases, I would accept it without any requests. However, I need to be careful about the removal and changes in the behavior because this PR touches existing aliases, which might have already users. Even if some existing aliases are something that not so many people are using, we do not remove them unless there is a clear problem.

[akinomyoga]

--clean only removes old slots so its functionality is limited, I don't believe it is widely used.

OK. But I still don't see a reason to remove ers. You just don't have to use it.

The code comments of aliases er and ers seem to tell that the difference between the two implies that the former is --clean and the latter is --depclean, so if anything would be changed, er could be changed to use --clean because we seem to have a discrepancy between the code comments and the actual aliases. (Or does running emerge with --clean break the system?)

With this in mind I replaced -U with -N.

Thank you for the description. I'm now convinced that we switch to -N.

Also, I think -j should be added by the user themselves because emfu should focus on completing its own functions.

These are aliases. It's different from an independent command-line tool. It's subtle whether we require the aliases to be focusing on a specific function with a plain setting. Aliases are intended to encapsulate the frequently used set of flags. There is no clear boundary on whether we accept or reject specific additional flags, but many aliases include additional flags that are just useful, such as -i for rm, etc.

For example, I define my own alias that essentially behaves like alias m='make -j' because I always specify -j to make. If anything is broken by specifying -j, it means that Makefile is broken in the sense that it doesn't correctly specify the dependencies. If the idea of the -j option of emerge is the same, I'd guess always specifying -j is not supposed to cause issues. But I'm not familiar with it, so I need an explanation about it.

Seeing the explanation so far, I still suggest keeping -j. It's been there, I didn't receive an issue report caused by it, and the reason for your change seems a philosophical one but not the one to solve an actual issue.

[blackteahamburger]

The code comments of aliases er and ers seem to tell that the difference between the two implies that the former is --clean and the latter is --depclean, so if anything would be changed, er could be changed to use --clean because we seem to have a discrepancy between the code comments and the actual aliases. (Or does running emerge with --clean break the system?)

I don't believe the diffirence between er and ers is --clean and --depclean. They both mean removing packages, while --clean removes old slots (which is already included in --depclean), so changing er to --clean is not appropriate. And this won't maintain backwards compatibility because er won't be able to remove packages, which will be confusing. Their difference is that ers removes "system packages", which I think may be packages in @system, which is strange since --depclean can be used to remove all packages. So I recommend removing ers. If you want to maintain backwards compatibility, I think ers can be marked as "deprecated" (e.g. emit a message when ers is executed?) and it can be removed after a while.

Seeing the explanation so far, I still suggest keeping -j. It's been there, I didn't receive an issue report caused by it, and the reason for your change seems a philosophical one but not the one to solve an actual issue.

What you said makes sense. But there is another variable MAKEOPT for specifying parallel make jobs. Gentoo wiki says:

When using parallel emerges (--jobs), the effective number of jobs run can grow exponentially (up to make jobs multiplied by emerge jobs). This can be worked around by running a localhost-only distcc configuration that will limit the number of compiler instances per host.

With this in mind I suggest -j should be added by the user themselves.

[akinomyoga]

I don't believe the diffirence between er and ers is --clean and --depclean. [...]

Thanks for your opinion. Then, let's not change er to use --clean. The suggestion was for the case when "if anything would be changed".

So I recommend removing ers. If you want to maintain backwards compatibility, I think ers can be marked as "deprecated" (e.g. emit a message when ers is executed?) and it can be removed after a while.

I still don't see the reason for removing it. It doesn't seem to cause problems based on the discussion so far.

When using parallel emerges (--jobs), the effective number of jobs run can grow exponentially (up to make jobs multiplied by emerge jobs). This can be worked around by running a localhost-only distcc configuration that will limit the number of compiler instances per host.

With this in mind I suggest -j should be added by the user themselves.

Thank you for those explanations. Does that mean specifying -j causes problems in general?

[blackteahamburger]

I still don't see the reason for removing it. It doesn't seem to cause problems based on the discussion so far.

It really doesn't cause problems, but it's a duplicate of er and is weird (What is a system package? If it's a package in system then it's exactly the same as er. If it's a package in @system, it's still redundant). So I think there's a strong case for removing it.

Thank you for those explanations. Does that mean specifying -j causes problems in general?

Not really. Gentoo wiki says:

When MAKEOPTS="-jN" is used with EMERGE_DEFAULT_OPTS="--jobs K --load-average X.Y" the number of possible tasks created would be up to N*K. Therefore, both variables need to be set with each other in mind as they create up to K jobs each with up to N tasks.

They just need the right combination.

By the way, there's also a varible EMERGE_DEFAULT_OPTS, which is another reason why -j shouldn't be added to emfu.

[akinomyoga]

It really doesn't cause problems, but it's a duplicate of er and is weird (What is a system package? If it's a package in system then it's exactly the same as er. If it's a package in @system, it's still redundant). So I think there's a strong case for removing it.

These seem just the reason for the ideal design. I wouldn't accept a change that breaks the backward compatibility just for the ideal design. The only reason that would allow to break the backward compatibility would be the situation where the original behavior would be a bug or cause some problems or conflicts.

Not really. Gentoo wiki says:

[...]

They just need the right combination.

That seems to imply that it would cause problems when used with a wrong combination? In particular, when the user doesn't set these variables, does specifying -j imply unlimited K and cause problems such as OOM? If that is the case, I would accept to drop the option -j. Or is portage clever enough to limit the number of the emerge jobs depending on the remaining memory? Anyway, I'm now becoming convinced that having -j by default would be tricky and wouldn't be so useful.

[blackteahamburger]

These seem just the reason for the ideal design. I wouldn't accept a change that breaks the backward compatibility just for the ideal design. The only reason that would allow to break the backward compatibility would be the situation where the original behavior would be a bug or cause some problems or conflicts.

Is backwards compatibility really that important? Even the C++ standard removes old facilities. And ers is not only weird, but also longer than er, so I think the impact of removing it is very low. As mentioned above, it could be deprecated first and then removed after enough time to minimize the impact.

That seems to imply that it would cause problems when used with a wrong combination? In particular, when the user doesn't set these variables, does specifying -j imply unlimited K and cause problems such as OOM? If that is the case, I would accept to drop the option -j. Or is portage clever enough to limit the number of the emerge jobs depending on the remaining memory? Anyway, I'm now becoming convinced that having -j by default would be tricky and wouldn't be so useful.

To be honest, I don't know which combination is the most correct.

Gentoo wiki says:

without an argument, -j will flood the processor with as many jobs as possible, and is not recommended

I just saw it. I used to think that -j without parameters was equivalent to -j$(nproc). It seems that it is scarier than I imagined.

There is a --load-average option for emerge to keep the load average of the system less, it "specifies that no new builds should be started if there are other builds running and the load average is at least LOAD (a floating-point number)". And also for make. But I just experienced oom-killer being triggered while compiling (with --load-average). Perhaps these options need to be adjusted flexibly, this article may be useful.

[akinomyoga]

These seem just the reason for the ideal design. I wouldn't accept a change that breaks the backward compatibility just for the ideal design. The only reason that would allow to break the backward compatibility would be the situation where the original behavior would be a bug or cause some problems or conflicts.

Is backwards compatibility really that important?

Yes.

Even the C++ standard removes old facilities.

This repository is not a standard but an implementation, meaning that this repository doesn't require other frameworks to copy everything including legacies. The C++ standard dropped some of the old facilities from the requirements on the conforming implementations, but that does not mean that the conforming implementations need to drop the feature. The implementations are always allowed to have extensions where the standard doesn't specify. It's the implementation's discretion whether to become pedantic on strict conformance or to be permissive.

The policy of this repository is to be careful in removing features. In the discussions so far, you didn't seem to care about the backward compatibility at all. However, even the C++ standard you mention cares about backward compatibility, so they survey the existing code bases before removing any features. Only when they confirm the feature is not widely used (or any real problems are found), the features would be dropped.

I should note that it's also related to the availability of human resources. We may follow a similar strategy as the C++ standard in principle, but unfortunately, there is no firm way to perform the survey for Oh My Bash (OMB). There is no way to know the exact size of the user base of OMB, but I estimate it to be the order of 100k users. Even though, it is hard to know the frequency of the uses of each alias under this large user base. Someone would need to perform a real survey on the users, but I doubt it's worth it. This project is not as valuable as you think.

Also, I'm not a user of OMB but merely a passive maintainer of OMB as discussed in #268. I started to be involved in OMB to handle the issues in ble.sh caused by OMB and took over the maintenance after a while. The users of OMB are not supposed to be as well-skilled as you think. OMB primarily targets users who are not familiar with Bash enough they configure the Bash settings by themselves. I'm really passive for this specific repository, so I wouldn't like to cause any unnecessary trouble that may puzzle the users. I have to say that this repository already contains tons of unreasonable legacies, but I wouldn't have time and wouldn't be willing to clean them up while handling the questions and reports from the users.

without an argument, -j will flood the processor with as many jobs as possible, and is not recommended

I just saw it. I used to think that -j without parameters was equivalent to -j$(nproc). It seems that it is scarier than I imagined.

That's good to know. Then, let's drop the option -j as it is in the PR.

[blackteahamburger]

In this case I would suggest adding a relevant reminder in the comments rather than leaving it as is to remove confusion as to why ers is retained since ers itself is weird.

[akinomyoga]

I agree with that. Could you add the commentary, and also you can comment that in aliases/README.md.

[blackteahamburger]

Thank you for your suggestions!

[akinomyoga]

Thank you for your quick responses and patience! I've merged it.