21 Apr 2021, 21:28

Exploring Template Literal Types

Another one in the short sharp blog posts that I am trying to do more of - this time on TypeScripts new Template Literal Types. Up until version 4.1, TypeScript had three literal types, namely strings, numbers and booleans. If you’re familiar with TypeScript there’s no doubt you use these every day, maybe something like this:

let maxPostLength: number;

// Later on you might do:
maxPostLength = 1000;

// Or

let postContent: string;

// Later on you might do:
postContent = "James here! Thanks for reading my blog post";

With the release of TypeScript 4.1, we introduce another literal type, namely Template Literals. I’m going to assume you have a basic to mid level understanding of TypeScript and work through some examples revolving around some ficitional user events code (i.e. pretend we’re building a social site!). Let’s take a look at a simple example:

type CreateEventType = "create";
type MessageEntityType = "message";
type MessageEntityEvent = `${EventType}-${EntityTypes}`;

If you’re familiar with template literals in JavaScript the syntax might look familar with the curly braces. Here MessageEntityEvent is equivalent to a string literal type of "create-message". This doesn’t give us much on it’s own, but if we start to mix it with string literal union types, you’ll start to see the magic:

type EventType = "create" | "read" | "update" | "delete";
type EntityTypes = "message" | "post" | "comment";
type EntityEvents = `${EventType}-${EntityTypes}`;

It might not be a initially obivious what we’ve done here, but the easiest way to explain why this is cool/useful is that EntityEvents is equivalent to a union of string literals like this:

type EntityEvents =
  | "create-message"
  | "create-post"
  | "create-comment"
  | "read-message"
  | "read-post"
  | "read-comment"
  | "update-message"
  | "update-post"
  | "update-comment"
  | "delete-message"
  | "delete-post"
  | "delete-comment";

I’m sure you’d agree that the equivalent string union types are firstly lengthy, but also harder to maintain. I say it’s harder to maintain because if we decide we want to add another entity type (maybe we add a like event to go with our posts), we have to add a line for each entity type. This being a manual process means we have to first write it out, but also that we could miss an event.

We can go beyond this though, as template literal types allow us to create combinations of union string literals; you can even union unions (inception!). Let’s say we add in the role of the given user into the event:

type EventTypes = "create" | "read" | "update" | "delete";
type EntityTypes = "message" | "post" | "comment";
type EntityEvents = `${EventTypes}-${EntityTypes}`;

type Roles = "admin" | "user";
type RoleEntityEvents = `${Roles}-${EntityEvents}`;

In turn this creates:

"admin-create-message" |
  "admin-create-post" |
  "admin-create-comment" |
  // etc
  "user-create-message
   "admin-create-post" |
  "admin-create-comment"
  // etc

There are actually 25 different combinations there, but I commented them to //etc to save your sanity. I’m hoping though here that the value is starting to show. So we can safely say template literal types save a lot of typing (forgive the double entendre). 25 handcraft string literal types vs 5 lines of using template literal types.

Now let’s look how we can mix this with other cool TypeScript features. Here we’ll use template literals types with generics and conditional typing. Specifically, here we’ll add some type safety via conditional typing to ensure that admins can fire admin events and regular users can’t:

type RegularEventTypes = "read" | "create" | "update";
type AdminEventTypes = "delete";
type EntityTypes = "message" | "post" | "comment";

type Events = `${RegularEventTypes}-${EntityTypes}`;
type AdminEvents = `${RegularEventTypes | AdminEventTypes}-${EntityTypes}`;

type AdminRole = "admin";
type UserRole = "user";

type Roles = UserRole | AdminRole;

interface User<T extends Roles> {
  role: T;
  fireEvent: (
    message: T extends AdminRole ? `${T}-${AdminEvents}` : `${T}-${Events}`
  ) => void;
}

// Later on

const admin: User<AdminRole> = {
  role: "admin",
  fireEvent: (event) => {
    if (event === "admin-delete-comment") {
      // Do something specific
    }
  },
};

const user: User<UserRole> = {
  role: "user",
  fireEvent: (event) => {
    if (event === "user-read-comment") {
      // Do something specific
    }
  },
};

This is pretty powerful, right? Alongside this useful addition, template literal types also provide a series of what they call Intrinsic String Manipulation Types, which essentially allow you to manipulate the strings within the template literals, doing things like uppercasing, lowercasing, capitalising the first letter (useful for camel case properties) etc. Here’s an example of how we could implement the Uppercase manipulation type into our example:

type EventTypes = "read" | "create" | "update";
type EntityTypes = "message" | "post" | "comment";

type Events = Uppercase<`${RegularEventTypes}_${EntityTypes}`>;

And now we’d get the equivalent of this:

type Events =
  | "READ_MESSAGE"
  | "READ_POST"
  | "READ_COMMENT"
  | "CREATE_MESSAGE"
  | "CREATE_POST"
  | "CREATE_COMMENT"
  | "UPDATE_MESSAGE"
  | "UPDATE_POST"
  | "UPDATE_COMMENT";

Quite useful if we want to selectively decide when to have event names upper case (passing to the server perhaps) and lowercase (using them as property keys for example).

Overall it was quite fun playing with the new feature and I look forward to using them day-to-day. I’d love to take them further and perhaps use them in conjunction with Mapped Types. The TypeScript site has quite a good section on this which I’ll let you explore directy. Have fun!

31 Jan 2021, 21:28

Measuring the World with JavaScript

If I asked you to measure two points on a table, you might get out a ruler or a tape measure to tell me the distance between the two. However, what if I asked you to measure the points between two capital cities? Suddenly it becomes more complicated, not only just logistically but also in terms of assumptions we have to make about that measurement.

One big change, apart from just scale, is that we now have to account for the roundness of the world. Thankfully mathematicians who were far smarter than me have come up with formulas that can account for this in various ways. One such formula is the haversine formula. This is credited as far back as 1801 to Spanish astronomer and mathematician José de Mendoza y Ríos.

The Haversine formula is useful in a lot of spherical trigonometry problems, as it allows a person to determine what is called a great-circle distance between two given points on a sphere. In particular relevance to what we are looking at today, it is useful for determining the approximate distance between two latitude and longitudes on the globe.

When it comes to translating this to code Chris Veness, a UK based programmer has written a healthy amount of code in the space - a lot of it is featured in the popular geospatial library Turf.js. Here’s a stand-alone adaptation of some of his code for the Haversine formula, written in TypeScript:

export function haversineDistance(
  pointOne: { lng: number; lat: number },
  pointTwo: { lng: number; lat: number }
) {
  const toRadians = (latOrLng: number) => (latOrLng * Math.PI) / 180;

  const phiOne = toRadians(pointOne.lat);
  const lambdaOne = toRadians(pointOne.lng);
  const phiTwo = toRadians(pointTwo.lat);
  const lambdaTwo = toRadians(pointTwo.lng);
  const deltaPhi = phiTwo - phiOne;
  const deltalambda = lambdaTwo - lambdaOne;

  const a =
    Math.sin(deltaPhi / 2) * Math.sin(deltaPhi / 2) +
    Math.cos(phiOne) *
      Math.cos(phiTwo) *
      Math.sin(deltalambda / 2) *
      Math.sin(deltalambda / 2);
  const c = 2 * Math.atan2(Math.sqrt(a), Math.sqrt(1 - a));

  const radius = 6371e3;
  const distance = radius * c;

  return distance;
}

The Haversine formula suffers from one core problem, which is that it assumes that the world is round. Unfortunately for us, the world is not actually round, it’s closer to what we’d determine as an oblate spheroid; a sphere that is flattened at the poles. As the Haversine makes this assumption, it cannot be guaranteed correct to provide results with better than 0.5% accuracy.

This is where another formula comes in, Vincenty’s inverse formula. The formula comes from Thaddeus Vincenty, a Polish American geodesist who was with the U.S. Air Force. Vincenty’s inverse formula works with the idea that the Earth is an oblate-spheroid. This approach is more complex and is also iteration based, unlike the haversine formula.

Again, I’ve taken some of Chris’s code and made it standalone (you can just copy and paste this function). It works on the WGS84 ellipsoid, as this is probably the most common use case. Lets take a look at the code, again in TypeScript:

export function inverseVincentyDistance(
  pointOne: { lng: number; lat: number },
  pointTwo: { lng: number; lat: number }
): number {
  const toRadians = (latOrLng: number) => (latOrLng * Math.PI) / 180;

  const phiOne = toRadians(pointOne.lat);
  const lambda1 = toRadians(pointOne.lng);
  const phiTwo = toRadians(pointTwo.lat);
  const lambda2 = toRadians(pointTwo.lng);

  const wgs84ellipsoid = {
    a: 6378137,
    b: 6356752.314245,
    f: 1 / 298.257223563,
  };
  const { a, b, f } = wgs84ellipsoid;

  const L = lambda2 - lambda1; // L = difference in longitude, U = reduced latitude, defined by tan U = (1-f)·tanphi.
  const tanU1 = (1 - f) * Math.tan(phiOne),
    cosU1 = 1 / Math.sqrt(1 + tanU1 * tanU1),
    sinU1 = tanU1 * cosU1;
  const tanU2 = (1 - f) * Math.tan(phiTwo),
    cosU2 = 1 / Math.sqrt(1 + tanU2 * tanU2),
    sinU2 = tanU2 * cosU2;

  const antipodal =
    Math.abs(L) > Math.PI / 2 || Math.abs(phiTwo - phiOne) > Math.PI / 2;

  let lambda = L,
    sinLambda = null,
    cosLambda = null; // lambda = difference in longitude on an auxiliary sphere
  let sigma = antipodal ? Math.PI : 0,
    sinSigma = 0,
    cosSigma = antipodal ? -1 : 1,
    sinSqsigma = null; // sigma = angular distance P₁ P₂ on the sphere
  let cos2sigmaM = 1; // sigmaM = angular distance on the sphere from the equator to the midpoint of the line
  let sinalpha = null,
    cosSqAlpha = 1; // alpha = azimuth of the geodesic at the equator
  let C = null;

  let lambdaʹ = null,
    iterations = 0;
  do {
    sinLambda = Math.sin(lambda);
    cosLambda = Math.cos(lambda);
    sinSqsigma =
      cosU2 * sinLambda * (cosU2 * sinLambda) +
      (cosU1 * sinU2 - sinU1 * cosU2 * cosLambda) *
        (cosU1 * sinU2 - sinU1 * cosU2 * cosLambda);

    if (Math.abs(sinSqsigma) < Number.EPSILON) {
      break; // co-incident/antipodal points (falls back on lambda/sigma = L)
    }

    sinSigma = Math.sqrt(sinSqsigma);
    cosSigma = sinU1 * sinU2 + cosU1 * cosU2 * cosLambda;
    sigma = Math.atan2(sinSigma, cosSigma);
    sinalpha = (cosU1 * cosU2 * sinLambda) / sinSigma;
    cosSqAlpha = 1 - sinalpha * sinalpha;
    cos2sigmaM =
      cosSqAlpha != 0 ? cosSigma - (2 * sinU1 * sinU2) / cosSqAlpha : 0; // on equatorial line cos²alpha = 0 (§6)
    C = (f / 16) * cosSqAlpha * (4 + f * (4 - 3 * cosSqAlpha));
    lambdaʹ = lambda;
    lambda =
      L +
      (1 - C) *
        f *
        sinalpha *
        (sigma +
          C *
            sinSigma *
            (cos2sigmaM + C * cosSigma * (-1 + 2 * cos2sigmaM * cos2sigmaM)));
    const iterationCheck = antipodal
      ? Math.abs(lambda) - Math.PI
      : Math.abs(lambda);
    if (iterationCheck > Math.PI) {
      throw new Error("lambda > Math.PI");
    }
  } while (Math.abs(lambda - lambdaʹ) > 1e-12 && ++iterations < 1000);
  if (iterations >= 1000) {
    throw new Error("Vincenty formula failed to converge");
  }

  const uSq = (cosSqAlpha * (a * a - b * b)) / (b * b);
  const A = 1 + (uSq / 16384) * (4096 + uSq * (-768 + uSq * (320 - 175 * uSq)));
  const B = (uSq / 1024) * (256 + uSq * (-128 + uSq * (74 - 47 * uSq)));
  const deltaSigma =
    B *
    sinSigma *
    (cos2sigmaM +
      (B / 4) *
        (cosSigma * (-1 + 2 * cos2sigmaM * cos2sigmaM) -
          (B / 6) *
            cos2sigmaM *
            (-3 + 4 * sinSigma * sinSigma) *
            (-3 + 4 * cos2sigmaM * cos2sigmaM)));

  const distance = b * A * (sigma - deltaSigma); // distance = length of the geodesic

  return distance;
}

I’ll be honest, Vincenty’s formula is a bit lost on me, but it promises to provide more accurate results! In fact, its use is very common in geodesy because the expected accuracy is to 0.5mm on the Earth ellipsoid. Pretty cool right? The one downside is that it is computationally slower due to the increased number of operations and the iterative approach.

Hopefully, you’ve enjoyed this short blog post about how to measure distances on the Earth with JavaScript (…okay TypeScript). Let me know if you would be interested in follow-ups on similar topics!

25 Dec 2020, 18:28

Experimenting Producing AVIF Images with Node.js

I’ve wanted to experiment with writing shorter and easier to digest blog posts on interesting web topics. So, this is my first exploring the AVIF Image format!

AV1 Image File Format or AVIF is an exciting new image format that can work with the web. It has been developed by the Alliance for Open Media aiming to be an open-source and royalty-free image format. It already has adopters some big-name adopters like Netflix, which is interesting to see.

I got excited about the possibilities for the new format and had a little gander at some of the analysis by some people much smarter than me to see what the deal was. The first post I looked at was AVIF vs WebP by Daniel Aleksandersen, which found against a baseline of JPEG there was a median file size reduction of 50.3% compared to WebP’s 31.5% reduction. Similarly, Jake Archibald did a nice comparison for a photo of an F1 car where WebP came in 57.7% of the size, and AVIF an impressive 18.2% of the size.

I was intrigued myself and saw that Sharp recently merged AVIF support in their Node.js library so I decided to have a little experiment with the new format.

It’s important to state here this specific blog post is not about comparing file sizes or perceived quality of output, as that is a fairly exact art and you can read all about the pitfalls of attempting that in this great post by Kornel Lesiński. Rather, this post is more about how you can create an AVIF image using Node.js today.

To start with I found this nice high-resolution image of Saturn on Wikipedia and thought I’d use it myself as a test image.

After this I went ahead and created a new Node.js project:

mkdir avif-sharp
cd avif-sharp
npm init
npm install sharp

I then wrote a little code snippet that exports a WebP image and another in the new AVIF format:

const sharp = require("sharp");
const fs = require("fs");

fs.readFile("saturn-original.jpeg", (err, inputBuffer) => {
  if (err) {
    console.error(err);
    return;
  }

  // JPEG
  console.time("jpeg");
  sharp(inputBuffer)
    .jpeg({ quality: 50, speed: 1 })
    .toFile("saturn.jpeg", (err, info) => {
      console.timeEnd("jpeg");
      console.log("jpeg", info);
    });

  // WebP
  console.time("webp");
  sharp(inputBuffer)
    .webp({ quality: 50, speed: 1 })
    .toFile("saturn.webp", (err, info) => {
      console.timeEnd("webp");
      console.log("avif", info);
    });

  // AVIF
  console.time("avif");
  sharp(inputBuffer)
    .avif({ quality: 50, speed: 1 })
    .toFile("saturn.avif", (err, info) => {
      console.timeEnd("avif");
      console.log("avif", info);
    });
});

The code uses console.time and console.timeEnd to get logs on how long these processes took. Running the code created three files, one saturn.jpeg, one saturn.webp and one saturn.avif as anticipated. You can find the full API docs for AVIF in Sharp here if you want to play with the configuration options.

Results

On my reasonably well-specced work machine, it took 287ms to output the compressed JPEG, 1288ms to output the WebP image and 22508ms to produce the AVIF image. Again it’s important to note here that quality: 50 and speed: 1 are not an absolute and comparable parameters here so try not to read into them.

With regards to size, at these settings for the AVIF image was 43.7kb compared to WebP which was 149kb, so approximately 29.32% of the size of the WebP image. Compared to the compressed JPEG the AVIF image was 13.35% of the size.

As I mentioned, take these results with a pinch of salt and I’d defer to Daniel Aleksandersen’s analysis of images at the same DSSIM and also take a look at BunnyCDNs take on the format for a more qualitative take.

I hope this shorter format still sits well, and that the blog post has shown you how you can create AVIF images using Sharp. Till next time!