[Bug]: Self-hosted Coolify fails on Docker container startup with dynamic configuration and [email protected] key deletion.

[Avenze]

Error Message and Logs

docker-startup.log

Steps to Reproduce

1. Set up Coolify according to documentation for self-hosted Docker installation (manual install)
2. Start Coolify while installed [email protected] SSH keys
3. "Accidentally" overwrite SSH key for .ssh/authorized_keys
4. Attempt installing newly generated SSH key into [email protected] as well as the public [email protected]
5. Failure? Fails at "Could not setup dynamic configuration: [email protected]: Permission denied (publickey,password)."

Example Repository URL

No response

Coolify Version

v4.0.0-beta.380 (latest)

Are you using Coolify Cloud?

No (self-hosted)

Operating System and Version (self-hosted)

Debian 11/bullseye in Proxmox LXC

Additional Information

The primary issue with this is that Coolify fails to connect to the host that it is running on, and when it fails it for some reason deletes the [email protected] as well as the corresponding public key from the /data/ssh/keys directory. Everytime I try stopping the container, creating the keys manually, and then setting their permissions correctly, then try starting the container again, it fails as usual and the [email protected] keys are now missing from the keys directory.

The SSH daemon logs shows that Coolify attempts connecting with the correct key, but cancels the connection prematurely.

See this thread in the coolLabs discord server:
https://discord.com/channels/459365938081431553/1324528031997628528

[Avenze]

Here's a reproduction of the issue on the machine experiencing the issue:
https://www.youtube.com/watch?v=sTlGXFWbrm4

[Avenze]

Possible cause of the issue I believe, seems as if the [email protected] file under the data/ssh/keys folder is semi-related to the private_keys schema in the postgresql database for Coolify,

Will take a further look to see if I can solve this issue.

[Avenze]

The stored value under the private_key in the rows is a Base64 encoded string, decoding it provides the JSON table below, which shows to contain the IV, the MAC and the private key under the value field, so likely the issue for me was that I replaced the authorized_keys file and then simply generated a new key.

{
    "iv": "wOITwsud",
    "value": "oVQkQ8riMrzYHORZRo",
    "mac": "900b637fa8891fc04fa686",
    "tag": ""
}

And from there, the Database/Seeders/PopulateSshKeysDirectorySeeder removed the [email protected] files due to them being incorrect with what was stored on the server, so at this point, I have two paths to follow, either get a public key from the stored private key in the database, or figure out an IV or MAC from the currently installed authorized_key on the machine itself.

Of course, the easiest part is getting the public key from the stored private key in the database, as otherwise I'd need to figure out a way to get an IV and MAC from just the private ED25519 key, which won't work (cough, elliptic curve encryption, not symmetric).

I'll take a look at how the initializing vectors and MACs are generated for the fun of it, why not lol

[Avenze]

Issue solved, yet I am still unable to connect to the Coolify instance on port 8000, but simply removing the localhost key from the database private_keys table, and then creating both the [email protected] and the [email protected] files with your new values genereated a new one and it is no longer outputting the error message in the console at startup.

[Avenze]

Fixed the issue with not being able to connect, somewhere the port that was required for the Docker container changed from 80 to 8080, changed that in the docker-compose.prod.yml and now it works.