tokio_postgres client query awaits forever

[marco-viewpoint]

My first try with this library and although I can establish a connection with the DB running a query always locks up my lambda until the 10 minute timer runs out.
Last output is "query sent", client.is_closed() returns false, making the user/pw wrong results in a connection error instead as expected.

I tried different queries and different functions (query_one, query_raw, etc.).
The code runs in a AWS lambda and connects to a serverless Aurora Postgresql DB inside a VPC.

Is there any additional debugging I can do to find the root cause?

src/main.rs

let cert = fs::read("global-bundle.pem")?;
println!("file read");
let cert = Certificate::from_pem(&cert)?;
println!("cert created");

let connector = TlsConnector::builder()
		.add_root_certificate(cert)
		.build()?;
println!("connector built");
let connector = MakeTlsConnector::new(connector);
println!("connector created");

let connect_result = tokio_postgres::connect(
		"host=#####.us-west-2.rds.amazonaws.com port=5432 user=### password=### dbname=### connect_timeout=10",
		connector,
).await;
println!("connection finished");

if connect_result.is_err() {
		let error_msg = connect_result.err().unwrap().to_string();
		println!("Error: {}", error_msg);
		return Ok(json!({ "error": error_msg }))
}

let (client, _connection) = connect_result.unwrap();
println!("connection unwrapped");
println!("Client closed: {}", client.is_closed());

let db_response = client.query_one("SELECT name FROM workspaces WHERE id = 760", &[]);
println!("query sent");
let db_response = db_response.await;
println!("query finished");

Cargo.toml

[package]
name = "test"
version = "0.1.0"
edition = "2021"
autobins = false

[dependencies]
lambda_http = "*"
lambda_runtime = "*"
graphql_client = "*"
env_logger = "*"
serde = "*"
serde_json = "*"
ndarray = "*"
tokio = { version = "1", features = ["macros"] }
tracing = { version = "0.1", features = ["log"] }
tracing-subscriber = { version = "0.3", default-features = false, features = ["fmt"] }
tokio-postgres = "*"
postgres-native-tls = "*"
native-tls = "*"
openssl = { version = "0.10", features = ["vendored"] }
futures = "*"

[[bin]]
name = "bootstrap"
path = "src/main.rs"

[profile.release]
strip = true

[sfackler]

Some questions/comments:

1. Does any query succeed? For example SELECT 1.
2. Can you properly interact with this database locally outside of the lambda environment?
3. A packet capture of the traffic with the database would be helpful to diagnose.
4. Calling an async function like query_one just constructs a future value. So the query sent message is sent before any IO actually happens for that query.

[marco-viewpoint]

1. SELECT 1 fails as well.
2. The DB works fine with AppSync and Velocity resolvers, I can try a python lambda under the same circumstances just to be 100% sure.
   /edit: I can confirm this operation works in the same environment with the same permissions using python and psycopg2.
3. Since the DB is only accessible inside the VPC a packet capture might be tricky.
4. Noted, thank you!

[marco-viewpoint]

I set up a new, public accessible DB with as close to the same settings as possible, so I could perform a network capture.
This time the rust code runs locally, but has the same issues
failed query.txt
.

[sfackler]

It looks like the request is never sent to the server. Can you provide a complete, self contained crate that produces that behavior?

[ManishLapasi]

Hi! Were you able to find a solution to this?
I'm trying to deploy a lambda function that connects to AWS RDS using tokio-postgres as well, and query_one and the like don't work for me as well.

[yagodorea]

Hey, I was just going through the same problem, but in my case, my issue was that I was inserting some rows before querying, but didn't call finish() on the writer.

Something like this:

let writer = BinaryCopyInWriter::new(sink, types);
for (a,b,c,d) in data.iter() {
    writer.as_mut().write(&[&a,&b,&c,&d]).await?;
}
// <-- Should've called writer.finish().await?; here
let rows = client.query("SELECT * FROM test;", &[]).await?;
for row in rows {
    println!("{:?}", row);
}

Not sure what are the inner workings behind this but calling finish solved my issue. Maybe you have another transaction pending and need to finish it before running query.

[JJaviMS]

Hi, I have faced the same error and after a lot of time I found out what was happening.
According to the examples on the docs, when you connect to the database you have to spawn a thread that performs the connection.

// The connection object performs the actual communication with the database
// so spawn it off to run on its own.
tokio::spawn(async move {
    if let Err(e) = connection.await {
        eprintln!("connection error: {}", e);
    }
});

From the docs example

Maybe this should be documented also on the connect method because it does tell nothing about the connection struct and it is easy to forget about it.