Currently When performing a system upgrade, Yay will first refresh the
database then perform the repo and AUR upgrade. This allows Yay to add
some features such as better batch interaction, showing potential
dependency problems before the upgrade starts and combined menus
showing AUR and repo upgrades together.
There has been discussion that this approach is a bad idea. The main issue
people have is that the separation of the database refresh and the upgrade
could lead to a partial upgrade if Yay fails between the two stages.
Personally I do not like this argument, there are valid reasons to Yay
to fail between these points. For example there may be dependency or
conflict issues during the AUR upgrade. Yay can detect these before any
installing actually starts and exit, just like how pacman will when
there are dependency problems.
If Yay does fail between these points, for the previously mentioned
reasons or even a crash then a simple refresh will not cause a
partial upgrade by itself. It is then the user's responsibility
to either resolve these issues or instead perform an upgrade using
pacman directly.
My opinions aside, The discussions on the Arch wiki has reached
a decision, this method is not recommended. So to follow the decided
best practises this behaviour has been disabled by default.
This behaviour can be toggled using the --[no]combinedupgrade flag
It should be noted that Yay's upgrade menu will not show repo packages
unless --combinedupgrade is used.
--ask is no longer used when installing AUR packages, instead pass no
confirm when we know there are no conflicts and wait for manual
confirmation when there are.
This means that when there are no conflicts there should be no change in
behaviour and the user will not need to intervene at all.
The old behaviour can still be used with --useask.
Instead of doing all the AUR stuff just pass to pacman and return. No
need for any of Yay's stuff when there's no AUR involved.
Of couse everything before that still happens. Upgrade menu ect.
Clean build needs to happen before downloading pkgbuilds so that they
can be deletd before downloading.
Editing and diff viewing needs to happen after downloading the
pkgbuilds.
Prevously we asked to clean and edit at the same time. Then clean,
download pkgbuilds and open the editor.
This poeses a problem for diff viewing and editing. It's likley that the
user will see the diff and use that to decide if they want to edit the
pkgbuild. Using the current method, the user will be asked to view diffs
and edit before actually seeing any diffs.
Instead split cleaning diff showing and editing to three seperate menus
in the following order:
show clean menu
clean
download pkgbuilds
show diff menu
show diffs
show edit menu
edit pkgbuilds
Also each menu is seperatly enableable. By default only the diff menu is
shows. If the user wishes to clean build, edit pkgbuilds or disable
diffs then the user can use the --[no]{clean,diff,edit}menu flags. This
replaces the --[no]showdiffs flags.
Newly cloned packages already start out at origin/master, so there is no
diff to show. Track if we cloned a package and if so make sure to show
the full diff
This is what 5775e3..43d2a6 has been leading up to. Git fetch will be
called on all pkgbuilds, then the user is offered a chance to view the
diffs. If they choose to continue, merging happens. This allows users to
abort the install after viewing diffs and still be able to see thoes
diffs again if they try to install later on.
This also makes the git stuff a little more modular which should help in
organzing diff showing + pkgbuild editing.
The order of targets does somewhat matter. For example doing something
like 'pacman -S db1/foo db2/foo' should cause the second package to be
skipped.
The order of targets also effects in which order they are resolved. This
should make errors more reproducable if any ever occur.
Prepare ends up getting ran twice every time we install a package,
theres not problems with doing so apart from a little inefficiency.
Previously the install flow would be like this:
downlod sources + verify
prepare + pkgver bump
full build (prepare included)
Now on the last point pass no extract to use the srcdir from the
previous command and pass noprepare and holdver because we allready did
these steps previously.
diff viewing can be toggled via --[no]showdiffs. When enabled diffs will
be shown for packages between the current HEAD and upstream's HEAD.
Packages downloaded via tarballs will be shown in full using the editor
git diff is used to show diffs. Therefore the pager for diffs can be
set via the PAGER and GIT_PAGER enviroment variables.
These flags limit operations to only check the repos or only check the
AUR. These flags apply to -S, -Si and -Su.
-a may also be used as a short option for --aur. --repo has no short
option as -r is taken.
Pacman 5.1 removes the symlink to the current directory for built
packages. This causes Yay to break for people who have set an external
PKGDEST.
Pacman 5.1 also brings an improved --packagelist option. This makes
it much simpler to find where packages will be placed. Hence this fix
also simplifies the code.
Yay has an -Sc option to clear it's cache. If using an external PKGDEST
this is now mostly useful for clearing out old pkgbuilds and sources.
paccache should be used for cleaning build packages.
The previous warning system would show warnings recursivley for all
packages being resolved. While I like this, other have complained at it
being overly verbose.
Either way the main purpose of this is to allow warnings to be printed
before the upgrade menu shows. This is mostly just to get a usable
warning system.
This may change if a better solution is found.
Targets are used for tracking wether a package should be marked as
explicitly installed or as a dependency. This is not ideal because you
can have a target such as java-environment that resolves to a different
package.
Therefore Targets are now used only for the initial dependency resolving
and checking for missing dependencies. The Explicit set is now used to
mark what packages are explicit, seperate from the targets.
The warnings were moved down to after the upgrade menu, mainly because
it is a lot easier to do this way, it may get moved back if it can be
done in a non hacky way,
Sort the provider menu alphabetically. Always ensure direct matches show
up first. This ensures hitting enter for the default value will always
be the same package that the user/dependency requested if an exact match
exists.
If a package is already installed pick that instead of providing a menu.
Ensure duplicates do not show up in the menu.
MakeOnly would be set to true when moving from normal deps to make deps
But would incorrectly stay set to true when moving to the deps of the
following packages.
depOrder.Aur now only holds one package from each base like
depCatagories does.
If --ignore was specified on the command line and the user skips
packages using the number menu, packages would not be properly skipped
because they the manual --ignore would overide the --ignore from the
menu.
Now correctly combine both --ignore flags into a single combined flag
when passing to pacman.
This bump reflects the big change introduced with using git cloning.
Therefore we know all versions pre-6 do not use git clone
Signed-off-by: Jguer <me@jguer.space>
Use git clone over tarballs for pkgbuild downloading during -S. This
option can still be toggled using the config flags.
The config option for selecting clone or tarball will be overiden if an
existing package is cached. The method used to download the package
perviously will be used regardless of the config.
Previously we ran pkgver() right after dowloading sources. This is
a problem because prepare() should be called and all dependencies
should be installed before pkgver().
Instead bump the pkgver while building then get the new pkgver used for
install. Previously we parsed `makepkg --printsrcinfo` to get the new
version. Insead use `makepkg --packagelist` as it is much faster.
Ensure aurWarnings will always be printed out in one block
use '->' for printing aur warnings and ignored upgrades
use '->' for conflict printing
use '->' for key importing
Say PGP keys not GPG keys
Add back green for input prompts
Use 4 spcaces over \t
Before `yay -Syu` called `pacman -Sy <pkgs to upgrade>`
We then later switched to it calling `pacman -Syu` this lead to yay
seeing no targets to when it was upgrading a bunch of packages it
assumed they must be deps. Correct this by adding repo packages to the
targets list.
Also ensure we dont mark packages as dependencies if they are already
installed. For example we install `foo` which requires `bar>5` but we
only have `bar=4` installed. In this case installing `foo` will pull bar
in as a dependency but it should not be marked as such because it
already exists.
This means that menus are now printed in noconfirm mode, I don't see
this as a problem because Pacman still prints its questions during
noconfirm.
When the user has edited pkgbuilds Yay will prompt if they want to
continue with the intall. This prompt is also enabled during noconfirm
to ensure the user is happy with the pkgbuilds.
To know what AUR packages need updating a rpc request is needed for all
packages. The dep tree is designed to cache everything to minimize the
amount of rpc requests. The downside of this is the dep tree ends up
with all sorts of packages in cache that it doesn't need. Then the
deptree tries to resolve deps for all of thoes packages.
By spliting the sysupgrade from the dep tree this stops this from
happening, it uses one more rpc request but also may lower the amount of
total rpc requests needed lated on.
This fixes a couple of tiny bugs such as triggering providers prompts
and printing AUR out of date messages for packages that are not going
to be installed.
This also fixes another display bug where repo packages from -Su would
not apear when printing the packages to be installed under [Repo].
--redownload is meant to only download the targets the user provides.
If the user enters aur/foo then Yay will find the package foo from the
aur, --redownload will see thats not what the user entered and skips the
download.
This makes it so after the dep searching is done, all db/ prefixes are
dropped.
This commit extends the conflict checking a lot, it adds support for:
Conflicting with provides as well as actual package names
Reverse conflicts
Inner conflicts
Both normal conflicts and inner conflicts are run in parallel.
Messages are now printing when checking conflicts.
This also fixes packages sometimes being listed as conflicting with
themselves.
The inner conflict is a little verbose and could be toned down a little
but I am insure exactly how to tone it down.
Previosly during `yay -Su` Yay would pass
`pacman -S <packages that need upgrade>` to pacman.
Instead pass `pacman -Su --ignore <number menu choices>`
This allows yay to handle replaces and package downgrades `-Suu`
When pkgbuilds are built by makepkg, if the pkgbuild's arch=() array
does not include the current carch set in makepkg.conf, makepkg will
fail to build the package.
Now, Yay detects if a pkgbuild does not support the arch set in
pacman.conf Yay will ask the user about this and ask them if they want
to build anyway, passing `--ignorearch` to makepkg.`
Note that Yay will check against the arch set in pacman.conf which is
what pacman uses to only allow installs of package of that arch. makepkg
will still use carch set in makepkg.conf to build packages. These two
values are expected to be the same otherwise Yay will fail.
The on disk .srcinfo is needed for this as the user should be asked pre
source download. This and pgp checking both use the on disk .srcinfo so
it is no longer a one off. Store the 'stale' srcinfos so they can be
accesed by both functions.
Fix typo where adding to has instead of depStrings
Error correcly when missing packages
Also handle cases where a package is provided multiple times. If one
package provies `foo=1` and another provides `foo=2` before the latter
would just overide the former version. Now both versions will be checked
against.
With the addition of pgp key checking in Yay, the srcinfo parsing was
moved to before the pkgver() bump, leading to outdated pkgbuild
information.
Srcinfo parsing must be done after the pkgver() bump
The pkgver() bump must be done after downloading sources
makepkg's PGP checking is done as the sources download
yays PGP importing requires the srcingo to be parsed
Quite the chicken and egg problem
It is possible to skip the integ checks after the sources download
then parse the srcinfo
do the yay PGP check
then run the integ checks
the problem here is that to generate the srcinfo `makepkg --printsrcingo` is ran
This causes the pkgbuild to be sourced which I am not comftable with
doing without the integ checks.
Instead we parse the on disk .SRRCINFO that downloads with the PKGBUILD
just for the PGP checking. Later on we parse a fresh srcinfo straight
from `makepkg --printsrcingo`. This is a little bit less efficient but
more secure than the other option.
The callback is set to allways silently say yes, When passing to pacman
for the intall pacman will then ask the question giving the user
a chance to answer.
Similar to the --redownload flag, when specifed targets will be rebuilt
even if an up to date version is cached. --rebuildall can be used to
ensure uninstalled dependencies are rebuilt as well.
Additionally, unlike --redownload there is also --rebuildtree. This
causes a rebuild and reinstall of a package and all of it's dependencies
recursivley. This is designed for when a libary updae, breaks an
installed AUR package due to a partial upgrade. polybar is a common
example
--rebuild allows you to easily skip the cache and rebuild against a newer
libary version. --rebuildtree is a more nuclear option where you can
rebuild the whole dependency tree.
When building a package from the AUR for which there are missing keys,
yay will now prompt the user whether it should try to import such keys
using gpg:
[...]
:: Parsing SRCINFO (1/3): libc++ (libc++abi libc++)
:: Parsing SRCINFO (2/3): aurutils
:: Parsing SRCINFO (3/3): cower
==> GPG keys need importing:
487EACC08557AD082088DABA1EB2638FF56C0C53, required by: cower
11E521D646982372EB577A1F8F0871F202119294, required by: libc++ (libc++abi libc++)
B6C8F98282B944E3B0D5C2530FC3042E345AD05D, required by: libc++ (libc++abi libc++)
DBE7D3DD8C81D58D0A13D0E76BC26A17B9B7018A, required by: aurutils
==> Import? [Y/n]
[...]
Default is to try to import the problematic keys ([Y/n]).
Adds the ability to pick which database to install a package from. This
is extended to also support for AUR packages. For example `extra/git`
and `aur/yay` should both work`. When not explicitly requesting
a database repo packages will be choosen over the AUR.
This features extends to yogurt mode, listings where a package shows up
in multiple database/the AUR is now handled.
The aur does not have a real pacman databse like core, extra ect. But
can be accessed as if was one with `aur/name`. Using Yay with a pacman
repository named "aur" is undefined.
If a pkgbuild is already in cache and matches the version on
the aur skip the download.
The version we check comes from the .SRCINFO file on disk which is never
updated. (updates through pkgver() edit the pkgbuild but do not effect
the .SRCINFO). Therefore if the the version of the .SRCINFO matches the
AUR's version there must not be an update.
In the case of the on disk version being newer than the AUR version we
can assume user interaction and they probably do not want it overwitten
so in that case also skip the download.
Use the command `git ls-remote <url> <branch>` to track devel updates
rather than relying on the GitHub API.
This allows devel update to work for every git based source and
elimantes the rate limiting from GitHub.
The yay_vcs.json format has changed to better support packages which
multiple vcs sources and to track the protocols each source uses. And
track the branch that each source tracks in it's fragment.
In a very specific case where the user runs `yay -Syu` then uses the
number menu to ignore all AUR upgrades after the Repo install the user
will still be prompted to install and download packages.
Add ( and ) to "installed" to match groups and out of date
Show popularity as well as voted in yogurt mode
Show download and install size in yogurt mode
Remove printing of white and black
Fix incorrect message on number menu
Yellow is now almost never used
Use `v` instead of `r` when printing version
show when a page was marked out of date on search and info
Previously Installing repo targets and repo dependencies of aur targets
was done in two steps.
doing `yay -S repo1 repo2 aur1 aur2` would lead to yay calling
`pacman -S repo1 repo2`. Then after we find all the dependencies of the
aur packages we call `pacman -S --asdeps <all repo dependencies>`. This
was an easy way to correctly mark dependencies. Since 005635b4 Both
these calls were merged into one command but dependency marking was
forgoten about.
Now correctly mark the dependencies through `pacman -D`
This allows the user to abort the install by telling the editor to exit
with a non zero status. e.g. `:cq` in vim. This should also catch errors
if the editor does actually fail or if the configured editor does not
exist.
Save the VSC Info as soon as the package install finishes. This should
ensure the VSC db does not end up in an incorrect state if an install
fails or is cancelled by the user.
This also adds better support for split packages. When one or more
packages are installed from the same base each individual package is
added to the db not just the base. This allows us to track individual
updates from the same base so that if one package gets updated we don't
assume all packages in the base are updated.
The -s was kept aroung because the package base `python-virtualfish`
failed to build without it. I first blamed this on some aur rpc bug
because it was missing some deps that were listed in python-virtualfish.
As is turned out python-virtualfish actually does some things wrong in
it's package build and if it was formated correctly this wouldnt be
a problem.
I kept the -s in just so it would build even though it might have had
some side affects. makepkg not respecting the --dbpath for example.
From now on python-virtualfish will fail to build when you dont have all
the deps installed. This is their fault and will not be fixed here.
Exit after parsing srcinfo with db do it actually does something.
when using gendb dont bother generating srcinfos.
Improved the formatting for some things mainly downloading and parsing.
This commit mostly focuses on merging the install() and upgradePkgs()
functions to be more integrated. Instead of both making their own aur
queries they now both combine theyre needes into one query. This gives
us a speed up and allows us to easily print orphands, misising and out
of date in one clean block.
Renable conflict checking, was disabled for testing, forgot to reenable
Use pacman --ask to remove conflicting packages, dont do it directly.
Say what each package conflicts with
Check for conflicts before aking for clean build
Reverse conflict are still not checked
Split packages are now grouped together when printing displaying the
package base and the packages inside of the base to beinstalled. If only
one packge is to be installed from a base and the package name matches
the base name print normally
Only build and install once per package base
Only ask questions once per package base for editing pkgbuils and clean
build
Install all deps for aur packages after the user confirms they want to
continue installing. This takes most of the load off of makepkg -s but
the -s is still left in for some edge cases with split packages.
When installing a package might be a dependancy for something and a make
dependancy for something else. This means when prompted to remove make
dependencies yay might also try to remove a package that is actually
needed causing a pacman error.
Adding the -u option will cause pacman to skip needed packages and give
a nice warning as it does so. It does not fix the root issue but works
for now.
I have replaced the old install and dependancy algorithms with a new
design that attemps to be more pacaur like. Mostly in minimizing user
input. Ask every thing first then do everything with no need for more
user input.
It is not yet fully complete but is finished enough so that it works,
should not fail in most cases and provides a base for more contributors
to help address the existing problems.
The new install chain is as follows:
Source info about the provided targets
Fetch a list of all dependancies needed to install targets
I put alot of effort into fetching the dependancy tree
while making the least amount of aur requests as
possible. I'm actually very happy with how it turned out
and yay wil now resolve dependancies noticably faster
than pacaur when there are many aur dependancies.
Install repo targets by passing to pacman
Print dependancy tree and ask to confirm
Ask to clean build if directory already exists
Download all pkgbuilds
Ask to edit all pkgbuilds
Ask to continue with the install
Download the sources for each packagebuild
Build and install every package
using -s to get repo deps and -i to install
Ask to remove make dependancies
There are still a lot of things that need to be done for a fully working
system. Here are the problems I found with this system, either new or
existing:
Formating
I am not so good at formatting myself, I thought best to
leave it until last so I could get feedback on how it
should look and help implementing it.
Dependancy tree
The dependancy tree is usually correct although I have
noticed times where it doesnt detect all the
dependancies that it should. I have only noticed this
when there are circular dependancies so i think this
might be the cause. It's not a big deal currently
because makepkg -i installed repo deps for us which
handles the repo deps for us and will get the correct
ones. So yay might not list all the dependancies. but
they will get installed so I consider this a visual bug.
I have yet to see any circular dependancies in the AUR
so I can not say what will happend but I#m guessing that
it will break.
Versioned packages/dependencies
Targets and dependancies with version constriants such
as 'linux>=4.1' will not be checked on the aur side of
things but will be checked on the repo side.
Ignorepkg/Ignoregroup
Currently I do not handle this in any way but it
shouldn't be too hard to implement.
Conflict checking
This is not currently implemented either
Split Paclages
Split packages are not Handles properly. If we only
specify one package so install from a split package
makepkg -i ends up installing them all anyway. If we
specify more than one (n) package it will actually build the
package base n times and reinstall every split package
n times.
Makepkg
To get things working I decided to keep using the
makepkg -i method. I plan to eventually replace this
with a pacman -U based method. This should allow passing
args such as --dbpath and --config to aur packages
aswell as help solve some problems such as the split
packages.
Clean build
I plan to improve the clean build choice to be a little
more smart and instead of check if the directory exists,
check if the package is already build and if so skip the
build all together.
This bug was caused by me not thinking when passing flags to aurInstall.
Currently a bunch of functions take an array of flags but we don't really
use them any more after the argument parsing update. These should be
refactored out eventually but I'm holding off until I'm more sure about
how these functions should look.
Argument parsing now works mostly as expected for repo packages.
AUR packages are a little tricky becauce makepkg cant handle args such
as '--dbpath'.
Also out alpm handle does not read the commandline options so any
arguments relient on alpm will be ignored.
For now though it seems yay has gained back the functionality it once
had. While also having improved argument handling which should also be
expandable and make it easier to handle anything new that might have
been missed.
This reimplemens all operations yay previously supported:
'-S' 'Syu' 'Si' ect.
Currently the argument objects are not fully implemented with the code.
Theres alot of funky conversion from
argument object -> pkg, flags -> argument object
This is to just get back to the functionally we had before (almost).
I have not looked into it yet but alot of the time pacman flags get
passed to makepkg. this cases an error for most commands now because the
new system Passes all flags:
`yay -Syu` -> flags = '-S' '-y' '-u'
while the old system would have done:
`yay -Syu` -> op = '-Suy', flags = ''
So extra flags are no longer passed at all currently.
This means:
'yay -S aic94xx-firmware --noconfirm -b /tmp/pacutilesu2q6hw/tmp-pacman -d'
will no longer error and 'aic94xx-firmware' will be installed but the
database path change will not apply and the dep checking will not be
skipped.
passToPacman now takes and argParser as a paramater. And is implemented
for the simple cases in cmd.go. Although passToPacman is now left non
working in places which still try to usr the old call format and will
need to be reimplemented.
