Josh Bavari's Ramblings

tech babbles on Ruby, Javascript, Rails, Phonegap/Cordova, Grunt, and more

Custom JSON Encoding in Phoenix

less than a 1 minute read

I recently have been working a lot using Leaflet.js to do some mapping.

In some of my models, I use the lovely Geo package for Elixir point and geospatial usage. I needed to add support for Poison to encode my model.

I’ve been serving geo json from my models, and I needed a way to use the JSON encoding way easier. I’m sending some data out to a ZeroMQ socket so I need to encode it by transorming my Geo module in a way that I could encode it with Geo JSON.

I modified my model in two ways – one by putting the @derive directive to tell Poison to encode only certain fields. That is one way.

In another way, I needed to encode it everytime by calling the Geo.JSON.encode method without me having to do it. You can see that in the defimpl.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
defmodule MyApp.Point do
  use MyApp.Web, :model

  # Option 1 - specify exactly which fields to encode
  @derive {Poison.Encoder, only: [:id, :name, :geo_json]}
  schema "points" do
    field :name, :string
    field :position, Geo.Point
    field :geo_json, :string, virtual: true

    timestamps
  end

  def encode_model(point) do
    %MyApp.Point{point | geo_json: Geo.JSON.encode(point.position) }
  end

  defimpl Poison.Encoder, for: MyApp.Point do
    def encode(point, options) do
      point = MyApp.Point.encode_model(point)
      Poison.Encoder.Map.encode(Map.take(point, [:id, :name, :geo_json]), options)
    end
  end
end

Cheers.

Adding Additional Static Paths in Phoenix

less than a 1 minute read

Phoenix is awesome.

A problem I ran into lately is how to add additional static paths to be served.

If you take a look in your lib/endpoint.ex file, you’ll see the plug used for adding static paths:

1
2
3
plug Plug.Static,
  at: "/", from: :electronify, gzip: false,
  only: ~w(css fonts images js favicon.ico robots.txt)

I wanted to add another folder to be served, ‘zips’, that I had to edit the only: line in the plug specification as such:

1
2
3
plug Plug.Static,
  at: "/", from: :electronify, gzip: false,
  only: ~w(css fonts images js favicon.ico robots.txt zips)

There you have it, now I can access the files in the zips folder in priv/static/zips through the URL. Cheers!

Shipping Data With Protocol Buffers in Elixir

less than a 1 minute read

Lately, I’ve needed some data shipped across to various nodes to exchange data in a variety of places on a problem I was working on. There were a few ways to get that data shipped across, as the usual suspects are JSON, XML, or Google’s Protocol Buffers.

For this specific problem, we were needing to get that data shared from C++ nodes to Elixir/Erlang.

Since the team was using Protocol buffers already, I decided to give them a run in Elixir using exprotobuf.

Note: the client for this experiement is on github.

The idea

The idea here is – we’ll capture pieces of data from one node and ship it to the server for processing. We define the data structure by a .proto file, then turn our data into binary form by encoding it, and finally shipping it to it’s destination. We could do the same thing with JSON, but we want the data as light as possible.

We’ll use ZeroMQ to ship the data and use the Elixir package exzmq to encode in protocol buffers.

The process

First we define our protocol buffer format for an image message we want to send with data, its width, height, and bits per pixel:

1
2
3
4
5
6
message ImageMsg {
  optional bytes data = 1;
  optional int32 width = 2;
  optional int32 height = 3;
  optional int32 bpp = 4;
}

We set up our application to use exprotobuf in our mix.exs file:

1
2
3
4
def application do
    [applications: [:logger, :exzmq, :exprotobuf],
     mod: {Zmq2, []}]
end

as well as including it as a dependency:

1
2
3
4
5
6
defp deps do
  [
    {:exzmq, git: "https://github.com/zeromq/exzmq"},
    {:exprotobuf, "1.0.0-rc1"}
  ]
end

Finally we create an Elixir struct from this proto file as such:

1
2
3
defmodule Zmq2.Protobuf do
  use Protobuf, from: Path.wildcard(Path.expand("./proto/imagemsg.proto", __DIR__))
end

Now that we have our protobuf file read in, let’s get an images binary data, create an elixir structure from our protobuf file, and send that data over a Zero MQ socket (using exzmq):

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
def check_file(file_path, socket) do
  IO.puts "Sending image from file path: #{Path.expand(file_path, __DIR__)}"

  case File.read(Path.expand(file_path)) do
    {:error, :enoent} ->
      IO.puts "No file at the path: #{file_path}"
    {:ok, img_data} ->
      send_image_data(socket, img_data)
  end
end

def send_image_data(socket, img_data) do
  img_message = Zmq2.Protobuf.ImageMsg.new(data: img_data)
  encoded_data = Zmq2.Protobuf.ImageMsg.encode(img_message)

  IO.puts "The encoded data: #{inspect encoded_data}"

  Exzmq.send(socket, [encoded_data])

  IO.puts "Sent request - awaiting reply\n\n"

  # {:ok, r} =
  case Exzmq.recv(socket) do
    {:ok, r} -> IO.puts("Received reply #{inspect r}")
    _ -> {:error, "No Reply"}
  end

end

And there we have it, a message sent serialized with protocol buffers. We can now apply this same strategy over any different protocol buffer messages we define, and ship them over any protocl we’d like.

Some inspiration

Along the R&D process, I came across David Beck’s blog. David has an experiment where he was sending several million messages in TCP where he explores some ultra-efficient methods of sending messages, it’s a great read. He also covers zeromq and protocol buffers that goes more in depth into Protocol buffers and some lessons learned.

Alas, we move on!

Cheers

Scoreboard Forms in React and Angular 2

less than a 1 minute read

As a developer, you should be focused on spending some of your own time learning and staying up to date with technology that is always moving.

I wanted to find a reason to hop into some of the ‘newer’ front-end frameworks, React and Angular 2, as well as some of the module bundlers browserify and webpack.

I had the opportunity to try out Angular 2 while it was still in alpha. With the recent announcement of Angular 2 going out in Beta, I wanted to build a Scoreboard form that went along with my Scoreboard project to evaluate the two frameworks.

This post will aim to build a simple scoreboard form in both frameworks, so you can see the same DOM interactions and the code it takes to form them.

Please also note, I’m still very much learning, and some code may not be ‘ideal’.

We’ll cover:

  • The scoreboard form idea
  • Learning the ideas behind the frameworks
  • The bundling systems
  • Angular 2 implementation (TypeScript)
  • React implementation (ES6)
  • The differences between the two
  • Pros and Cons of each (in my eyes)

All of the source code is available on my Github profile in the scoreboard-form repository.

The Scoreboard Form

A scoreboard is simple – you’ll enter the two team names, then specify a touchdown or field goal for either team.

That means we will need a few components: a scoreboard and a team.

The idea will be to build these components in React and Angular2, having them use similar templates to render to the equivalent DOM structures.

Learning the ideas behind the frameworks

Both frameworks aim to contain all of the functionality and display into a component.

The idea will be to build a team component, that displays teams, and a scoreboard component, that will display both of those teams and have a submit method to post that data to our scoreboard API server.

The main difference we will see between the two frameworks is adapting to ES6 or TypeScript.

In either framework, we will create a class for the component in ES6/TypeScript, and then connect the template to it, and finally attach it in the DOM.

The bundling systems

We will use Browserify to pack up React into a single module, while using webpack to bundle up Angular 2 into a single bundle.

What is bundling you say? It’s taking all the source for our components and bundling them up with the framework code to have one JavaScript file to load the entire bundle. That way, we only need one JavaScript file instead of a whole load of <script> tags.

Angular 2 implementation

Angular 2 is built in TypeScript, which is a superset of JavaScript that allows types to have a ‘stronger type’ to work with. We will build our component in TypeScript, and transpile it down to ES5 JavaScript.

Building the Team component

In Angular 2, we need to use a decorator (see this blog post about TypeScript decorators) to specify a Team as a component that will render.

We will import the Decorator, Component, and then apply it to our class. The Component decorator specifies which DOM element it will attach to (in this case, team), any inputs our control may have, what template it will use (specified as the template key), and what other directives will be used to parse our template.

Then, we have our class that defines the component itself with a name and score, methods to increase score (touchdown and fieldGoal), a toJson method, and finally callbacks to update itself from the parent component.

The team component:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
import {Component, EventEmitter, Output} from 'angular2/core';
import {NgFor, NgIf, NgModel} from 'angular2/common';

@Component({
  directives: [NgFor, NgIf, NgModel],
  selector: 'team',
  template: `
    <div *ngIf="name == ''">
      <h3>No team set</h3>
      <input type="text" [(ngModel)]="nameInput" placeholder="Enter a name"/>
      <button type="submit" (click)="setName()">Set name</button>
    </div>
    <div *ngIf="name != ''">
      <h3></h3>
      <button (click)="touchdown($event)">Touchdown</button>
      <button (click)="fieldGoal($event)">Field Goal</button>
      <h4>Score: </h4>
    </div>
    `
    }
})
export class Team {
  @Output() updateTeam = new EventEmitter<Team>();
  constructor() {
    this.nameInput = '';
    this.name = '';
    this.score = 0;
  }

  fieldGoal(e) {
    e.preventDefault();
    this.score += 3;
  }

  touchdown(e) {
    e.preventDefault();
    this.score += 7;
  }

  setName(nameInput) {
    this.name = this.nameInput;
    this.nameInput = '';
    if(this.updateTeam) {
      this.updateTeam.next(this);
    }
  }

  toJson() {
    return { name: this.name, score: this.score };
  }
}

Defining the scoreboard component

Now we need to displays these teams in a side by side manner, a callback to update information from the team component, and a method to submit the scores to the API.

We’ll define the component as follows:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
import {Component} from 'angular2/core';
import {Team} from '../team/team';

@Component({
  directives: [Team],
  selector: 'scoreboard',
  template: `
    <form (ngSubmit)="submitScore()">
      <div class="row">
        <div class="col-md-6">
          <h2>Home Team</h2>
          <team (updateTeam)="updateHomeTeam($event)" home="true"></team>
        </div>
        <div class="col-md-6">
          <h2>Visitor Team</h2>
          <team (updateTeam)="updateVisitorTeam($event)"></team>
        </div>
      </div>
      <div class="row">
        <button type="submit">Submit</button>
      </div>
      <div *ngIf="submitted">
        JSON payload: 
      </div>
    </form>  
  `
})
export class Scoreboard {
  homeTeam: Team = new Team();
  visitorTeam: Team = new Team();
  submitted: boolean = false;
  jsonPayload: string = null;

  constuctor() {
  }

  submitScore() {
    this.submitted = true;
    this.jsonPayload = JSON.stringify({ homeTeam: this.homeTeam.toJson(), visitorTeam: this.visitorTeam.toJson()});
  }

  updateHomeTeam(team: Team) {
    this.homeTeam = team;
  }

  updateVisitorTeam(team: Team) {
    this.visitorTeam = team;
  }
}

Pros and Cons

Pros

Cons

  • Angular 2 – docs are all over the place.
  • Main blogs that are linked from docs site are using old kabob style (e.g. *ng-if instead of *ngIf).
  • Webpack configuration – I didn’t include zone.js in my entries, and I could not get any DOM updates coming from my components changing.
  • When to use two-way bindings, and one-way bindings was good enough.
  • No ‘why’ to what im doing – it aims to just follow the same ‘idea’ as Angular.js.
  • Plunkers aren’t up to date.

React implementation (ES6)

Now that we have the basic idea of the team and scoreboard, you’ll see React is very similiar. Instead of having a decorator specify the template and DOM elements to attach to, we’ll specify a class that extends React.Component, a method that will render the markup, and finally, some bootstrap code to attach our class to a DOM element.

Defining the team component

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
import React from 'react';

export default class Team extends React.Component {
  constructor(props) {
    super(props);
    this.props = props;
    this.name = props.name;
    this.score = props.score || 0;
    this.setName = this.setName.bind(this);
    // this.state = {name: this.name, score: this.score};
    this.touchdown = this.touchdown.bind(this);
    this.fieldGoal = this.fieldGoal.bind(this);
  }

  fieldGoal(e) {
    e.preventDefault();
    this.score += 3;
    this.setState(this);
  }

  touchdown(e) {
    e.preventDefault();
    this.score += 7;
    this.setState(this);
  }

  setName(e) {
    e.preventDefault();
    this.name = this.refs.teamName.value;
    this.setState(this);
    this.props.update(this);
  }

  toJson() {
    return { name: this.name, score: this.score };
  }

  render() {
    if (!this.name) {
      return (
        <div>
          <h3>No team set</h3>
          <input type="text" ref="teamName" placeholder="Enter a name.." value={this.props.name}/>
          <button onClick={this.setName}>Set Name</button>
        </div>
      );
    } else {
      return (
        <div>
          <h3>{this.name}</h3>
          <button onClick={this.touchdown}>Touch Down</button>
          <button onClick={this.fieldGoal}>Field Goal</button>
          <h4>Score: {this.score}</h4>
        </div>
      );
    }
  }
}

Defining the Scoreboard component

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
import Team from './team.jsx';
import React from 'react';

export default class Scoreboard extends React.Component {
  constructor(props) {
    super(props);
    this.homeTeam = {};
    this.visitorTeam = {};
    this.url = this.props.url;
    this.submit = this.submit.bind(this);
    this.updateTeam = this.updateTeam.bind(this);
    this.submitted = false;
    this.jsonPayload = null;
  }

  submit(event) {
    event.preventDefault();
    this.submitted = true;
    this.setState(this);
    this.jsonPayload = JSON.stringify({ homeTeam: this.homeTeam.toJson(), visitorTeam: this.visitorTeam.toJson()});
  }

  updateTeam(team) {
    if (team.props.home) {
      this.homeTeam = team;
    } else {
      this.visitorTeam = team;
    }
  }

  render() {
    var jsonInformation = this.submitted ? (<div>JSON payload: {this.jsonPayload}</div>) : null;
    return (
      <form onSubmit={this.submit}>
        <div className="row">
          <div className="col-md-6">
            <h2>Home Team</h2>
            <Team home="true" update={this.updateTeam}></Team>
          </div>
          <div className="col-md-6">
            <h2>Visitor Team</h2>
            <Team update={this.updateTeam}></Team>
          </div>
        </div>
        <div className="row">
          <button type="submit">Submit</button>
        </div>
        {jsonInformation}
      </form>
    )
  }
}

Now you’ll see, theres no way we tell React to attach to a DOM node to attach our components to the browser DOM.

This happens by the bootstrapping code:

1
2
3
4
5
6
7
import React from 'react';
import ReactDOM from 'react-dom';
import Scoreboard from '../component/scoreboard.jsx';

window.app = (function() {
  return ReactDOM.render(<Scoreboard/>, document.getElementById('react-scoreboard'));
})();

Now, React knows to use our Scoreboard component (the one that was imported) to attach it to the react-scoreboard DOM element with an id of react-scoreboard. Internally for the Scoreboard, it specifies it’s render method:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
import Team from './team.jsx';
// .. snipped code ..
render() {
  var jsonInformation = this.submitted ? (<div>JSON payload: {this.jsonPayload}</div>) : null;
  return (
    <form onSubmit={this.submit}>
      <div className="row">
        <div className="col-md-6">
          <h2>Home Team</h2>
          <Team home="true" update={this.updateTeam}></Team>
        </div>
        <div className="col-md-6">
          <h2>Visitor Team</h2>
          <Team update={this.updateTeam}></Team>
        </div>
      </div>
      <div className="row">
        <button type="submit">Submit</button>
      </div>
      {jsonInformation}
    </form>
  )
}

Pros and Cons

Pros

React Dev tools – inspect react components, super handy. Dev docs talk about how to think in react – giving the why before the what, really helped understand the concepts.

Cons

Dev tooling is not straight forward – you have to decide yourself. Figuring how to plug in rendering steps between state changes. this.setState({}) with some state information.

Differences between the two

The main difference I can see if how Angular 2 specifies its selector to find out how it attaches to a DOM element you specify.

React just follows using JSX to specify the component, which you can pass in properties.

Angular 2 takes the approach of keeping state and doing stateful checks from its Virtual DOM diff’ing. However, the templating directives you can use, like *ngIf requires handling a template of some sort, where as React, you can just use JavaScript conditionals to render your DOM.

Conclusions

I really like the approach React takes. I also feel like it is a year early to the Virtual DOM party, and Angular 2 is really trying to keep up.

As far as intuition and ease of development goes, React was definitely easier. Even with my previous Angular 2 knowledge, it still took me longer to get up and going.

To give Angular 2 a fair shot, it is still in Beta. However, if I were to start a project today, it would be in React, due to the huge community that is building, the tooling available, and being backed by Facebook, one of the utmost leaders in User inface design and performance.

I hope this short write up helps! If you have any questions, please drop a comment and we’ll clear things up!

As a reminder, here is all of the code is available on Github, feel free to open an issue.

Cheers!

Using Brew to Install Old Versions

less than a 1 minute read

I just wanted to share a quick little tidbit on how to install older brew versions.

I was having some issues with an older version of Elixir failing tests (1.0.1), and the latest version (1.1.1) is working fine.

Just running brew install elixir gets latest.

To get 1.0.1 installed, I first went to the homebrew github repo, looked at the Library/Formula folder, found the elixir.rb formula to install elixir, looked in the history, found 1.0.1, and then executed the following line:

1
brew install https://raw.githubusercontent.com/Homebrew/homebrew/8506ced146655c24920f3cc5b20e6bc9e6e703cc/Library/Formula/elixir.rb

That did it, I easily got 1.0.1 installed, and going back to 1.1.1 was super easy.

Hope this helps, enjoy! Cheers!

Adding PostgreSQL Geometric Type Support to Elixir

less than a 1 minute read

In the last week or so, I’ve had a blast playing around with basic Postgres geometric types to do basic earth distance queries.

From my favorite blog, Datachomp shows how to use radius queries in postgres to find the closest place to get a burrito fix. Since I’ve been on an Elixir kick lately, I figured it was time to contribute back to the open source world by adding first class burrito, err, geometric type support.

Initial reaction

I immediately made an Ecto model trying to use the point type in my model:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
defmodule MyApp.LocationPoint do
  use MyApp.Web, :model

  schema "location_point" do
    field :name, :string
    field :date, Ecto.DateTime
    field :location, :point
    timestamps
  end

  @required_fields ~w(name date)
  @optional_fields ~w(location)

  @doc """
  Creates a changeset based on the `model` and `params`.

  If no params are provided, an invalid changeset is returned
  with no validation performed.
  """
  def changeset(model, params \\ :empty) do
    model
    |> cast(params, @required_fields, @optional_fields)
  end
end

Right away, when I ran the commands to retrieve this location from iex, it gave me some errors:

1
2
3
4
5
$ iex -S mix
$ alias MyApp.Repo
$ alias MyApp.LocationPoint
$ Repo.all(LocationPoint)
$ ** (ArgumentError) no extension found for oid `600`

Right away, I knew this mission was up to me to get point support into Postgrex.

In this post, I’ll outline how to add type support to Postgres via the Elixir package, postgrex. We will walk through adding the Point data type in Postgres.

This post will cover:

  • How to see how postgres stores its types (built in and composite)
  • How postgrex does its type lookups
  • Finding the source type – adding it to postgres senders
  • Looking up postgres source code for data mapping
  • Adding new type Point type
  • Adding built in Type structs
  • Adding encode method
  • Adding decode method

How Postgres stores its types

Postgres stores its types in a special system table called pg_type (docs). It defines a few things about the type:

  • Its typelem – how the type is stored – array, or otherwise
  • Its typsend – Output conversion function (binary format), or 0 if none
  • Its typarray – an oid to another array type that has its send method

How Postgrex does type lookups

Postgrex at it’s core is a simple data adapter into PostgreSQL from Elixir. It’s an awesome library, and if you’re using Ecto, you’re already using it!

First, let’s look at how they are loading most types, by looking them up in the pg_type table in postgres:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
  ### BOOTSTRAP TYPES AND EXTENSIONS ###

  @doc false
  def bootstrap_query(m, version) do
    {rngsubtype, join_range} =
      if version >= {9, 2, 0} do
        {"coalesce(r.rngsubtype, 0)",
         "LEFT JOIN pg_range AS r ON r.rngtypid = t.oid"}
      else
        {"0", ""}
      end

    """
    SELECT t.oid, t.typname, t.typsend, t.typreceive, t.typoutput, t.typinput,
           t.typelem, #{rngsubtype}, ARRAY (
      SELECT a.atttypid
      FROM pg_attribute AS a
      WHERE a.attrelid = t.typrelid AND a.attnum > 0 AND NOT a.attisdropped
      ORDER BY a.attnum
    )
    FROM pg_type AS t
    #{join_range}
    WHERE
      t.typname::text = ANY ((#{sql_array(m.type)})::text[]) OR
      t.typsend::text = ANY ((#{sql_array(m.send)})::text[]) OR
      t.typreceive::text = ANY ((#{sql_array(m.receive)})::text[]) OR
      t.typoutput::text = ANY ((#{sql_array(m.output)})::text[]) OR
      t.typinput::text = ANY ((#{sql_array(m.input)})::text[])
    """
  end

You can see that under the hood, we’re querying Postgres and asking it for it’s types, so we can do OID lookups and call the appropriate encoder/decoder methods. From here, we can match up our newly added types encoding/decoding methods.

Finding the source type – adding it to postgres senders

Find information about the geometrics category:

SELECT * from pg_type where typcategory = 'G';

We will see the point type has an oid of 600, which is using a send specification of point_send. Other notable send types for geometries: point_send lseg_send path_send box_send poly_send line_send circle_send.

Thus, we’ll update the send types in postgrex, located in the binary.ex file:

1
2
3
4
5
6
@senders ~w(boolsend bpcharsend textsend varcharsend byteasend
            int2send int4send int8send float4send float8send numeric_send
            uuid_send date_send time_send timetz_send timestamp_send
            timestamptz_send interval_send enum_send tidsend unknownsend
            inet_send cidr_send macaddr_send point_send
            ) ++ @oid_senders

Boom, that gets us the oid to encode/decode off of!

Looking up postgres source code for data mapping

I hopped into the Postgres source code and looked up the struct type for point, found here.

1
2
3
4
5
typedef struct
{
  double    x,
        y;
} Point;

Great, its just two floats, no big deal.

Adding the point struct

Let’s craft our Postgrex stuct type in builtins.ex then!

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
defmodule Postgrex.Point do
  @moduledoc """
  Struct for Postgres point.

  ## Fields
    * `x`
    * `y`
  """
  require Decimal
  @type t :: %__MODULE__{x: float, y: float}

  defstruct [
    x: nil,
    y: nil]
end

Adding the encode method

Now since we are sending PostgreSQL binary data, we need to take our data and map it to it’s binary form, via an encode method.

However, postgrex is going to do a type look up, based on the types that we used in that query above.

We’ll add the methods to encode, that does some pattern matching to decipher we are getting the correct sender value.

1
2
def encode(%TypeInfo{type: "point", send: "point_send"}, %Postgrex.Point{} = point, _, _),
  do: encode_point(point)

As you can see, we are encoding, when a TypeInfo tuple is passed with type point and send point_send! Great, we just pass that to this method to parse out the two floats passed in the binary object:

1
2
defp encode_point(%Postgrex.Point{x: x, y: y}),
  do: <<x::float64, y::float64>>

It just takes those two values, and serializes them down to their binary counterparts.

That now handles the test we’ve got to keep us honest:

1
2
3
test "encode point", context do
  assert [[%Postgrex.Point{x: -97, y: 100}]] == query("SELECT $1::point", [%Postgrex.Point{x: -97, y: 100}])
end

This test as promised, takes a Postgrex.Point type, and encodes it to the binary form, and sends it off to Postgres. How beautiful.

Adding the decode method

Now, when we get binary values from Postgres, we need to map that to our Point type we’ve created.

Adding the functions to decode in binary.ex:

1
2
3
4
5
6
7
8
def decode(%TypeInfo{type: "point"}, binary, _, _),
  do: decode_point(binary)

# ..snip..

defp decode_point(<<x::float64, y::float64>>) do
  %Postgrex.Point{x: x, y: y}
end

The real meat and potatoes is, receiving our binary parameter, mapping its individual segmets as two floats, sized 8 bytes, and then with the pattern matching mapping those to our Postgrex.Point struct. QED.

And the test:

1
2
3
4
test "decode point", context do
  assert [[%Postgrex.Point{x: -97, y: 100}]] == query("SELECT point(-97, 100)::point", [])
  assert [[%Postgrex.Point{x: -97.5, y: 100.1}]] == query("SELECT point(-97.5, 100.1)::point", [])
end

Conclusion

Once I finally figured out what pieces were what, I was able to run and create the point type, its mappings, and its senders it required, easily mapping to the struct in Elixir.

I plan to keep working on postgrex, to add first class support for Postgres geometric types.

Cheers!

The Scoreboard Project

less than a 1 minute read

Lately I’ve been wanting to dig more into some technologies I’ve been wanting to explore and gain more experience. Not only this, but I wanted to make sure my dev workflow was still improving, my tools were getting sharpened, and I was re-establishing the best practices as much as I could.

Those specific technologies I wanted to dig into was:

  • Building a CLI in Ruby, using Thor
  • A Sinatra Modular API
  • Solifying Sequel Model usage and JSON serialization
  • Building a dashboard using Dashing
  • Diving more into Rubocop for Ruby static code analysis
  • Automated Code Review using CodeClimate

I found a way to connect all the dots in what I’m calling the scoreboard project. I chose these technologies because it would let me shine up my ruby/sql skills without a framework carrying me the way. (Although they mostly did anyway!)

This blog post will go over the idea of making an API around scoreboards. There will be a simple CLI tool to gather scores on ‘games’. Those scores will be sent to the API, to store in Postgres. The dashboard project will then pull these values from postgres and display them in an easy to view manner.

This post

With this post, i’ll go over the individual pieces of the project, the interesting tidbits of each one, and finally go over a short retrospective over the entire project.

In entire time, the project took about a day and a half. It was fun, and I really enjoyed the experience over all.

The pieces

All of the projects are listed on my github profile. I’ve been trying to keep most issues in the github repo’s issue page for each respective project.

All of the projects are checked by CodeClimate, and I’ve been trying to keep up with rubocop’s rules on these as I go.

Scoreboard CLI

The idea for the CLI was to prompt the user for a home team and visitor team, then collect data about getting a touch down for the home team, for example: h:td.

It would keep prompting for more scoring plays until the user gives a SIGTERM or hits CTRL+D.

First I started by reading up on Thor, which was an absolute pleasure to work with. You can download it via gem install scoreboard.

To make it available via command line, I added this:

1
2
  spec.bindir        = "bin"
  spec.executables   = spec.files.grep(%r{^bin/}) { |f| File.basename(f) }

Then in bin/scoreboard link, we just require in our CLI and run it with the arguments:

1
2
3
4
5
6
7
8
9
10
11
12
13
#!/usr/bin/env ruby

require_relative "../lib/scoreboard/cli"

begin
  cli = Scoreboard::Cli.new
  cli.start(ARGV)
rescue => e
  raise e if $DEBUG
  STDERR.puts e.message
  STDERR.puts e.backtrace.join("\n")
  exit 1
end

A note on the SIGTERM exception handling

If you see in my STDIN.each_line loop where I read in scoring entries, you will see I rescue all Exception. This could be improved to find the exact SIGTERM exception is being thrown, but for simplicity, I left it catching the general exception.

Scoreboard API

The API has a few paths, based on the /api/v1 namespace for requests.

You can access its teams or the entire scoreboard, via GET /api/v1/teams or GET /api/v1/scores. You can see the core Sinatra Application on github.

It was absolutely easy to set up the Sequel migrations to define the team table and the scoreboard table in postgres.

The main tying point was getting the Sequel models to serialize, which was solved in another blog post.

Scoreboard Dashboard

Dashing was really easy to get started, a project set up, and out the gate.

First I had to include Sequel to get me my data, which I included an Api model to ease the SQL bridge for me.

The main point here was the scoreboard.rb file which was scheduled to run every 5 seconds, gather data from some crafty queries, and send that data to the dashboard. Other than the HTML markup, this was the chunky part of it:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
require 'sequel'


DB = Sequel.connect('postgres://localhost/scoreboard')
scoreboard = DB[:scoreboard]
team = DB[:team]

send_event('games-played', { value: scoreboard.count })

def teams_played
  DB[<<-eos
      select
        sum(value)::int as value,
        label
      from (
        select count(home_id) as value, name as label from team inner join scoreboard on team.id = scoreboard.home_id group by scoreboard.home_id, team.name

        UNION ALL

        select count(visitor_id) as value, name as label from team inner join scoreboard on team.id = scoreboard.visitor_id group by scoreboard.visitor_id, team.name
      ) sub
      group by value, label
      order by value desc
    eos
  ]

end

def team_scoreboard
  DB[<<-eos
      select 
        s.id,
        t.name as home_team, 
        t2.name as visitor_team, 
        home_score, 
        visitor_score 
      from team t 
      inner join scoreboard s on s.home_id = t.id 
      inner join team t2 on t2.id = s.visitor_id
      limit 10;
    eos
  ]
end

SCHEDULER.every '5s' do
  teams = teams_played.map do |item|
    {:label => item[:label], :value => item[:value]}
  end
  send_event('games-played', { value: scoreboard.count })
  send_event('teams', { items: teams })
  puts "Scoreboard: #{team_scoreboard.to_a}"

  send_event('scoreboard', { items: team_scoreboard.to_a })
end

Retrospective

  • What went right
  • What went wrong
  • What could be improved

What went right

  • The CLI came together smoothly. Thor was easy to get running.
  • Getting data to post to the API was a breeze
  • Sinatra and Sequel were easy to hoist up a simple API to take POST data and serve GET requests as JSON
  • Getting data into the dashboard was SUPER easy with Sequel, no need to do the ORM dance
  • Dashing was easy to create my own scoreboard component, using the data- type DOM attributes

What went wrong

  • Had some issues handling SIGTERM in CLI
  • CLI still doesnt validate input
  • API for Sinatra was a little difficult to get JSON serialization off the bat
  • Dashing is very ‘opinionated’ and doesnt give you more room to fit into an existing app
  • No tests were made
  • Nothing is deployed to servers yet

What could be improved

  • Minitest suite for CLI, API, and the Dashboard
  • Dashboard process tasks could be broken out to be more DRY
  • CLI needs to check and validate input
  • API needs to add in rollbar, new relic, or other metrics to help find errors
  • Deploy all the things!

Future plans

The plan is to keep working on this project and continue improving tooling and getting other best practices in place. Finally, ship it to digital ocean and enjoy the conveniences they provide.

Using JSON Serializers in Sinatra

less than a 1 minute read

I ran into a quick little issue with serializing some of my Sequel models.

The official JSON serializer docs are great, I just wanted to shine more light on the issue.

If you’re using Sequel models, just throw in the quick line of plugin :json_serializer.

1
2
3
4
5
6
7
8
# Get our database connection
require_relative "./db"
module ScoreboardApi
  class Team < Sequel::Model(:team)
    plugin :json_serializer
    serialize_attributes :json, :name
  end
end

Then, you just use the Sinatra contrib gem to have it json serializer attach:

1
2
3
4
# Web framework
gem "sinatra", "1.4.6", require: "sinatra/base"
# Sinatra addons - JSON, Namespace
gem "sinatra-contrib", "1.4.6"

Set up your API routes and spit out JSON:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
require "bundler"
require "sinatra/base"
require "sinatra/json"
require "sinatra/namespace"

require "./models/scoreboard"
require "./models/team"

Bundler.require

module ScoreboardApi
  class App < Sinatra::Application
    register Sinatra::Namespace
    configure do
      disable :method_override
      disable :static

      set :sessions,
          :httponly     => true,
          :secure       => production?,
          :expire_after => 31557600, # 1 year
          :secret       => ENV["SESSION_SECRET"]
    end

    use Rack::Deflater

    namespace "/api/v1" do
      get "/scores" do
        json :scoreboard => Scoreboard.all
      end

      get "/teams" do
        json :teams => Team.all
      end
    end
  end
end

That”s all folks!

Playing With Safari 9 Force Touch API

less than a 1 minute read

I just got the new Macbook Pro not too long ago and noticed they had this intimately named featured called Force Touch.

Force touch, as it sounds, lets you touch things more intimately. The gist is this – there are sensors on the new touch pad that let you determine how hard the touch pad is touched, and has some API into that.

I admit, the API is kinda just for fun since only Safari 9 and new Macbooks have the feature. Still, I wanted to play more with ES6 and create a little API wrapper to make it easier and let this feature name to be a little less creepy.

After a little googling, I found an awesome blog post by Joshua Kehn that detailed more about using Force touch. I wanted to make a quick wrapper around it to do some fun effects and animations.

The idea is – we’ll have an API called TheForce that we can attach handlers to for fun visual effects.

Joshua’s demo had a box that when tapped and applying pressure, padding would be increased. I took his gist and turned it into an ES6 module. (See the demo)

The API

I wrote it up last night in a quick hurry. The gist is this:

  • Attach to a DOM element
  • Apply a glow to the element based applied pressure to touchpad

Code to attach a button named you, and trigger it automatically:

1
2
3
var luke = TheForce.beWith('you').alter({'boxShadow');

<!-- luke.alter('boxShadow'); -->

Respect Yourself

less than a 1 minute read

I’m writing this post as a memoir for myself, a retrospect of sorts. I read Seth Godins blog about Unconscious consumption and that post inspired me to write this post.

In the last 3 years, amazing events have unfolded. I helped create a start up, RaiseMore. I moved to Boulder, Colorado. I got a job at Ionic. I met a strong, amazing, beautiful woman who is my girlfriend.

I trained jiu-jitsu 4-5 times a week, I kept working on my strength, and trying to inspire others to do the same. I was striving to always read self-help books, watching informational videos, TED talks, ignite talks, challenging and pushing myself, and helping others grow.

All these new and exciting things kept happening to me. Many times before, I’ve asked myself, ‘how did I get so lucky?’. Things just had a snowball effect and kept getting better and better!

I’d like to think what attracted all these amazing things into my life was from these advancements.

However, something happened during these last 4 months that really threw me in a rut. I developed a poor attitude. I started thinking I was a victim. I no longer work for the company that I had looked up to for so long.

As I sit here now and reflect on the last few months, I know I’ve made my fair share of mistakes. I take full responsibility for that. And due to some of those mistakes, I’m unemployed (and looking for opportunities!) and I know it’s my own fault.

I’d like to stop and take a minute to deep dive and retrospect on what happened. I know, deep in my heart, that it all changed when I stopped respecting myself and making my personal growth a priority.

I’m realizing that while all these good things were occurring, one thing remained constant – I put myself and my personal growth first.

I’d like to deep dive into what really happened, and discover the why.

Deep diving

Questions to ask:

  • What did you do right?
  • What did you do wrong?
  • What can you do better?

What did I do right?

In January, I was jumping head first into the Ionic CLI. I was familiar with Node.js, but wasn’t nearly at the level I wanted to be at. So I jumped head first into blogs, training courses, books, and more.

I learned a ton by failing forward, bouncing ideas off of others, and most importantly, maintaining a healthy work life balance.

In March, I came up with the crazy idea to make the Ionic CLI easier to consume by almost any user by just point/clicking. In my free time after working on the Ionic CLI, I started cranking out Ionic Lab – that would do just that.

It was so freeing to use my creative side and use my intuition, listening to myself as I craved to learn more about Node.js and Electron!

After demoing Ionic Lab to the Ionic team, they were convinced this was a tool that could open the door to more developers and really improve the tooling. Thus, I was instructed to work on it full time. Such a joy!

Then it hit, this thing had to ship. I had to really buckle down and work out the final kinks in integrating Cordova fully into Ionic Lab, which took me down rabbit holes into discovering more about Electron than I wanted to know, fixing stdio errors and things that occurred deep in Cordova library code.

Once we shipped out Ionic Lab in July, I was told to focus on Ionic 2. The rest of the team had been heads down working on the codebase since March, I was the new guy, so I had to really step it up and put in the time to learn Angular 2, TypeScript, ES6, Webpack, and all the other new ground to cover.

It was exciting once again to jump head first into learning and exploration. But something happened that I did not expect.. I became a cynic.

What did I do wrong?

I think it wasn’t until after July that things started to slow down and my personal happiness started to decline. I started working more hours (60 hours was the minimum). I stayed up late working most every night. I slept less. I started working out less (I gained 10-20 lbs). I started putting myself and my personal growth last.

I stopped calling my friends. I stopped reaching out to my support group as much. I wasn’t maintaining my happy upbeat attitude I’ve been told has been so inspiring.

I started putting work first. I started accepting others views of myself. I stopped valuing my work, my time, and my efforts to put that towards a company that at the end of the day, only wanted more and more. We all know what this leads to, burn out.

This all was boiling down to the AngularConnect conference, where we planned to unveil Ionic 2 to the European crowds. I was given a hefty task of learning all that was Ionic 2, adding cross platform functionality in Ionic 2, adjusting the Ionic CLI for v2, helping curate content for a workshop at the conference, as well as building the first Ionic v2 app, the Ionic conference app.

All this was going down and other life issues arise, as they do. My older (15y old) dog was getting worse by the day. My girlfriend was not happy that I was putting her second to work. My friends stopped calling me. My shoulder got injured. I lost confidence in my work because I was burning out.

Then, a personal goal I had set didn’t get delivered to a group of users I was really wanted to, the Windows Ionic CLI users.

This hurt. Failure when placed heavy, can hurt. It hurt even more when I was putting work first, and myself last.

Looking more hopefully at the situation..

What can I do better?

First off – be honest with myself. Tell others I need a helping hand when needed. Going forward, I will look out for myself better (we all know employers want employees that can look out for themselves).

Don’t accept others views of myself unless I first believe them!

Most importantly, respecting myself. I should always take the time to have a 1 on 1 with myself and really get a sense of where I’m at. If I need a break, use that vacation that we are so lucky to have!

We tend to forget that if we dont build ourselves up, the world surely wont. (Unless you are lucky enough to have awesome friends that stick through it thick and thin). Burnout happens when you don’t let your support structure help put out of the flame before you become a pile of ash.

So please, take this time to evaluate yourself and your needs. Be honest with yourself. Focus on caring for yourself. Be compassionate to yourself. Let yourself make mistakes, but learn from them.

Focus on the positive. Surround yourself with upbeat people, ditch the negativity, and remind yourself that your future is in your hands. Take it.

Don’t forget that the only difference between a victor and a victim, is their mindset.

Remember, there is no luck, only the work you put in and the effect you see afterwards.