nix411
January 19, 2022, 6:51pm
1
I'm trying to install prodigy using poetry but I get the following error (omitted everything after ? with ...)
400 Client Error: Bad Request for url: https://s3.eu-west-1.amazonaws.com/data.prodi.gy/dist/prodigy-1.11.7-cp310-cp310-linux_aarch64.whl?...
I'm also puzzled why it says cp310 since I'm running python 3.9.9 but maybe that's not related.
Init a poetry project with poetry init and add the following to pyproject.toml
[[tool.poetry.source]]
name = "prodigy"
url = "https://[LICENCE_KEY]@download.prodi.gy/index"
Then try to install prodigy by running poetry add prodigy.
If I run poetry run pip install -r requirements.txt with --extra-index-url set then it works just fine. A very notable difference in the url's requested is
https://s3.eu-west-1.amazonaws.com/data.prodi.gy/dist/prodigy-1.11.7-cp39-cp39-linux_x86_64.whl?....
versus the above.
ines
January 19, 2022, 7:31pm
2
Ah, it looks like you're unfortunately running into this known poetry issue with HTTP Basic Auth credentials in URLs (which is still open):
opened 04:04PM - 10 Sep 21 UTC
Bug
Triage
<!--
Hi there! Thank you for discovering and submitting an issue.
Before… you submit this; let's make sure of a few things.
Please make sure the following boxes are ticked if they are correct.
If not, please try and fulfill these first.
-->
- [x] I am on the [latest](https://github.com/python-poetry/poetry/releases/latest) Poetry version.
- [x] I have searched the [issues](https://github.com/python-poetry/poetry/issues) of this repo and believe that this is not a duplicate.
- [x] If an exception occurs when executing a command, I executed it again in debug mode (`-vvv` option).
<!--
Once those are done, if you're able to fill in the following list with your information,
it'd be very helpful to whoever handles the issue.
-->
- **OS version and name**: Ubuntu 20.04
- **Poetry version**: 1.1.8
- **Link of a [Gist](https://gist.github.com/) with the contents of your pyproject.toml file**: https://gist.github.com/kabirkhan/4bc063c148b28ada980f08a79cc5f35c
## Issue
My team has an s3-backed private PyPI server which is secured using a LICENSE_KEY. Users purchase a license and can use the key to download any past version of the .whl. The implementation of the server uses this LICENSE_KEY as the username in HTTP Basic Auth, with an empty password. If we can validate that LICENSE_KEY, we fetch a list of .whl files from an AWS S3 bucket for all past versions + platforms and return a pre-signed URL using the boto3 library.
If I pip install from this server:
```bash
pip install my_package -f https://{LICENSE_KEY}:@my_server.com
```
this installs the package correctly.
However, when I add this server as a secondary source in poetry:
```toml
[tool.poetry.source]
name = "private_pypi"
url = "http://TEST_LICENSE:@my_server.com"
secondary = true
```
the install fails with a 400 error. The error occurs due to poetry passing my HTTP Basic auth credentials to the AWS S3 pre-signed URL. AWS throws the error saying there can only be 1 auth method (which is the pre-signed URL) but Basic Auth credentials were also passed.
The error seems to occur due to the request Session being retained here:
**Repository passing requests.Session with basic auth creds** - [`repositories/legacy_repository.py`](https://github.com/python-poetry/poetry/blob/c320955007a36173245c24d16076f9cf06b334c9/poetry/repositories/legacy_repository.py#L449)
**Using provided session** [`helpers.py`](https://github.com/python-poetry/poetry/blob/c320955007a36173245c24d16076f9cf06b334c9/poetry/utils/helpers.py#L98)
I can definitely see scenarios where the auth credentials should be retained so it makes sense why the session is used here but I'd love the ability to turn this off, maybe as a setting in the `[tool.poetry.source]` section.
Since this works with a pip install I'd expect it to work with poetry as well.
Gist with the basics of my PyPI server are [here](https://gist.github.com/kabirkhan/4bc063c148b28ada980f08a79cc5f35c)
Happy to help contribute as well and thanks for the great project!
Thanks @willfrey for helping identify this bug.
In the meantime, you could pre-download the wheels and use Poetry to install from a path: Installing from wheels. · Issue #76 · python-poetry/poetry · GitHub
1 Like
nix411
January 20, 2022, 9:32am
3
I guess I could yes but then it wouldn't be platform agnostic, right? I need to support ubuntu, mac, mac m1 unfortunately.
ines
January 20, 2022, 4:20pm
4
I'm not super familiar with poetry but does it have something equivalent to pip's -f option that you can point to a local directory of files and that will automatically pick the best-matching wheel for the given platform? https://prodi.gy/docs/install#wheel
Another option would be to run a mini local PyPi index that serves the wheels – but that's of course a bit more involved.
1 Like
Update
The following appears to work now using Poetry version 1.2.0b3 on macOS version 12.5.1 (Intel):
poetry source add prodigy "https://download.prodi.gy"
poetry config http-basic.prodigy <personal license> ""
poetry add --source=prodigy prodigy
We haven't tested for different OS/Poetry versions but want to post in case it may help others.
1 Like
nix411
September 23, 2022, 9:47am
6
nix411
September 23, 2022, 1:29pm
7
Hi @ines
The poetry team requests access to the wheels to further investigate. Do you mind reaching out? See this comment . Obviously I won’t share with them - that’s for you to decide.
nix411
September 23, 2022, 8:45pm
8
ryanwesslen:
The following appears to work now using Poetry version 1.2.0b3 on macOS version 12.5.1 (Intel):
poetry source add prodigy "https://download.prodi.gy"
poetry config http-basic.prodigy <personal license> ""
poetry add --source=prodigy prodigy
We haven't tested for different OS/Poetry versions but want to post in case it may help others.
Turns out I had to do
poetry config http-basic.prodigy <personal license>
and then just confirm an empty password. Otherwise I got 403 Client Error: Forbidden for url: https://s3.eu-we...