Notes on trying to learn Rust doing web development

ToDo:
  1. The code is broken because of the < and > symbols for generics. Welcome to 2019...
  2. The code snippets are a bit sad. They come with horizontal scrollbars. I'd prefer some sort of hard wrapping on mobile if necessary. Fixed it by just making things narrow enough so that it works on my desktop and phone. Will have to do for now.
  3. I don't know how to CSS. I copy&pasted some stuff but the font is too small on my phone. :(
  4. There are a bunch of questions in code comments. I should collect those here.
On this page I will collect my notes while trying to learn a bit of Rust by doing some simple web development. Well, hopefully at some point it will become less simple. The level is "I know how to program. I have read a Rust tutorial. I don't know any advanced topics."

Part 1: How do we even web at all?

To get started I was looking at arewewebyet which wasn't particularly helpful in picking something. I get that peope might want to keep it neutral but that makes it a lot less useful. After some more digging I settled on warp for now. Getting the warp example up and running was easy. I just ran 
cargo new warptest
and searched for warp on crates.io and added 
warp = "0.1.15"
to the Cargo.toml and the example code from the github repo to the main.rs 
use warp::{self, path, Filter};

fn main() {
    // GET /hello/warp => 200 OK with body "Hello, warp!"
    // smon: I am not quite sure if I like macros or not.
    // They make this kind of DSLish thing easier which often makes things
    // more concise but also often makes them hard to read.
    let hello = path!("hello" / String)
        .map(|name| format!("Hello, {}!", name));

    warp::serve(hello)
        .run(([127, 0, 0, 1], 3030));
}
at this point cargo run starts up a webserver and we can access localhost:3030/hello/world.

Part 2: How do we send and receive data?

warp actually comes with a nice todo-list example which is supposed to do exactly what we want. Because I am trying to understand what is going on and I didn't really need the features I just started copy-pasting parts of it together. I ended up with several steps, the first one being to introduce the new logger pretty-env-logger by the same author as warp.

log stuff!

extern crate pretty_env_logger;
// smon: aparrently this crate is not enough we also need the next one
#[macro_use]
extern crate log;

// smon: and we need this for the environment variables
use std::env;

fn main() {
    // smon: I guess you can control this logger with environment variables.
    if env::var_os("RUST_LOG").is_none() {
        // Set `RUST_LOG=todos=debug` to see debug logs,
        // this only shows access logs.
	// smon: I am changing the name here. I wonder what that means.
	// Does a log statement need to include the namespace?
        env::set_var("RUST_LOG", "warptodo=debug");
    }
    pretty_env_logger::init();
    // smon: How does this statement know that
    // the "warptodo" definition should apply to it? Does it?
    info!("Hello!");
}

Really sending and receiving data!

Okay, now I am actually adding most of the code for that todo example... 
extern crate pretty_env_logger;
#[macro_use]
extern crate log;
#[macro_use]
extern crate serde_derive;

// smon: and we need this for the environment variables
use std::env;

// smon: I definitely have no clue what this stuff is. Something about
// threads and sharing data efficiently and such...
use std::sync::{Arc, Mutex};
use warp::{Filter, http::StatusCode};

// smon: Dear god...yeah, no clue!
type Db = Arc>>;

#[derive(Debug, Deserialize, Serialize)]
struct Todo {
    id: u64,
    text: String,
    completed: bool,
}

fn main() {
    if env::var_os("RUST_LOG").is_none() {
        // Set `RUST_LOG=todos=debug` to see debug logs,
        // this only shows access logs.
        env::set_var("RUST_LOG", "warptodo=debug");
    }
    pretty_env_logger::init();
    info!("Hello!");

    let db = Arc::new(Mutex::new(Vec::::new()));
    let db = warp::any().map(move || db.clone());

    let todos = warp::path("todos");
    let todos_index = todos.and(warp::path::end());

    // When accepting a body, we want a JSON body
    let json_body = warp::body::content_length_limit(1024 * 16).and(warp::body::json());

    // `GET /todos`
    let list = warp::get2()
        .and(todos_index)
        .and(db.clone())
        .map(list_todos);

    // `POST /todos`
    let create = warp::post2()
        .and(todos_index)
        .and(json_body)
        .and(db.clone())
        .and_then(create_todo);
    // smon: What is the difference between map and and_then?

    let api = list.or(create);

    let routes = api.with(warp::log("warptodo"));

    warp::serve(routes).run(([127, 0, 0, 1], 3030));
}

fn list_todos(db: Db) -> impl warp::Reply {
    debug!("listing all todos");
    // Just return a JSON array of all Todos.
    warp::reply::json(&*db.lock().unwrap())
}

fn create_todo(create: Todo, db: Db) -> Result {
    debug!("create_todo: {:?}", create);
    Ok(StatusCode::CREATED)
}
So there are quite a few things here which I don't understand yet. However, before getting into that I wanted to be able to test the receiving function, which requires sending a JSON request.

Testing by sending automated requests

Now, my first instinct would have been to do this in Python. Because I know it's very easy. Actually, this is how easy it is: 
import requests
url = 'http://localhost:3030/todos'

r = requests.get(url)
print(r)
print(r.json())

jd = { 'foo': 42 }
r = requests.post(url, json=jd)
print(r)

jd = { 'id': 42, 'text': 'do stuff', 'completed': False }
r = requests.post(url, json=jd)
print(r)

r = requests.get(url)
print(r)
print(r.json())
But since we are trying to learn Rust, maybe we should do this part in Rust as well. Once again, we turn to seanmonstar and his reqwest library. The docs are pretty nice so changing the example to send the proper JSON body wasn't too difficult. Two things that confused me for a while are
  1. Parsing JSON from a response uses the variable you are trying to assign it to do infer what the format of the JSON has to be. So the line from the example 
    let resp: HashMap = reqwest::get("https://httpbin.org/ip")?.json()?;
    doesn't work because we are getting back an empty array and we cannot convert that into a map. Even more confusing, removing the type annotation from the resp variable gives it the unit the type () which means we now get an error unless the returned JSON is empty. However, the following works, at least for empty arrays. ;) 
    let resp: Vec = reqwest::get("http://localhost:3030/todos/")?.json()?;
  2. warp does the same things on the server side. I tried sending some pretty random JSON and was confused that the create_todo function never gets called. However, it is pretty clear from the fact that the function receives a Todo object that parsing must happen before it gets called. I just missed that.
Anyway, here is the test code I eventually cooked up: 
extern crate reqwest;

fn main() -> Result<(), Box> {
    let resp: Vec = reqwest::get("http://localhost:3030/todos/")?.json()?;
    println!("{:#?}", resp);

    let new_todo_json = r#"{
                "id": 23,
                "text": "foo",
                "completed": false
                }"#;
    let resp = reqwest::Client::builder()
        .build()?
        .post("http://localhost:3030/todos/")
        .body(new_todo_json)
        .send()?;
    println!("{:#?}", resp);
    Ok(())
}

Part 3: Securing data with HTTPS

Okay, yes, we're only sending data to localhost but we are sending things and so I feel the need to turn on HTTPS. Which warp and reqwest both support. Of course I still don't know how to turn it on. It turns out the code change in warp is actually quite minimal: 
//warp::serve(routes).run(([127, 0, 0, 1], 3030));
warp::serve(routes)
  .tls("/tmp/sslpublic.pem", "/tmp/priv.key")
  .run(([127, 0, 0, 1], 3030));
However! I couldn't figure out how to actually turn on the TLS feature. TLS support is an optional feature of the library and I had never used an optional feature before. So I screwed that up in some creative ways. Here's what you need in your Cargo.toml: 
warp = { version = "0.1.15", features = ["tls"] }
On the reqwest side of things we pleasantly don't have to do anything except change the URL to https://. And of course you need to generate the certificate files and have them in the right format. I probably should have written down the commands because it was easy and boring but it still took a while to collect the necessary information. Oh, and if we use self-signed certificates we actually do have to change a bunch of things in the reqwest part to trust our self-signed certificate because otherwise we will get an SSL error. The following works, but I have no clue if it is the right way to do it or the nice way to do it. 
extern crate reqwest;

use std::fs::File;
use std::io::Read;

fn main() -> Result<(), Box> {
	let mut buf = Vec::new();
	File::open("/tmp/sslpublic.pem")?
		.read_to_end(&mut buf)?;
	let cert = reqwest::Certificate::from_pem(&buf)?;
    let client_builder = reqwest::Client::builder()
        .add_root_certificate(cert)
        .build()?;

    let resp: Vec = client_builder.get("https://localhost:3030/todos/").send()?.json()?;
    println!("{:#?}", resp);

    let new_todo_json = r#"{
                "id": 23,
                "text": "foo",
                "completed": false
                }"#;
    let resp = client_builder.post("https://localhost:3030/todos/")
        .body(new_todo_json)
        .send()?;
    println!("{:#?}", resp);
    Ok(())
}