BRAD GESSLER: All right. So, while these are,
these responses are coming in.
It looks like quite a few people have
either handcrafted html and css applications.
I'm not sure if they're single-page application,
or just deploy straight-up Rails applications
or all the logic on the server-side.
So that's good. I, I hope you'll learn
something new today about how my company,
Poll Everywhere, which the,
the product is what you're seeing.
We're a software as a service company.
We just sell this to presenters based on audience size.
So, I'm gonna fire up the slides here. All
right. My display settings are working, so this is
good. All right, so Middleman. The missing view in
the Rails API stack.
Let's just jump into it. So, as I said
before, I am Brad Gessler, CTO and co-founder of
Poll Everywhere. We started this company about six years
ago now. So, we were mobile first, whatever that
meant, SMS. So we went through all the pains
of having to upgrade, if you will, to the
second vintage of mobile first, which is mobile, basically
a mobile website. We had a lot of fun
doing that and, and learning things the hard way.
But first, what is middleman? How many of you
have heard of Jekyll in this room? That seems
to be the, the predominant static website generator. And
if you go to the Ruby toolbox and you
ask Ruby toolbox for, hey, what should I use
to generate a static website? You're going to get
Jekyll. But it's, it's not like Jekyll, in the
sense that it uses a lot of the more
modern, basically RubyGems inside of what you see in
Rails 4 and Rails 3, which is, the fact
that it uses tilt to manage all of the
template and, and sass and all that good stuff.
So, if you're looking for a tool to build
a static site, don't be scared off by the
fact that middleman is catching up with Jekyll. And
you should feel a lot better about that after
this talk.
So, middleman is extraordinarily well-documented. Thomas Reynolds built middleman.
He's the, the guy behind this. And he also
built an amazing website with a lot of great
documentation behind it. So if you go to, I
believe it's middlemanapp dot com, you can get a
sense of, of everything that, that this static site
generator involves.
Another really nice thing about this static site generator
is that it's, it's very modular. So it actually
uses rack in its gut, so if you understand
rack, you can write a lot of extensions using
rack, which is kind of crazy. A static site
generator using rack to, to do certain things.
And there's also a, an extension framework, where you
can roll things out, like, a middleman blog, if
you so please. So, middleman blog in middleman is
essentially a, a drop and replacement for Jekyll. With
a few other things. If you're used to writing
GitHub-flavored markdown, there's a few tweaks that you have
to make there to, to get that whole thing
working.
So, getting started with middleman is actually very similar
to what it's like getting started in a Rails
application. You install the gem, you initialize the application,
it creates all of this boilerplate code. You can
see from this directory structure you have your configuration
file, default index.html.erb. You get a layout file and
then you have all of your assets. Stylesheets and
JavaScripts. And then you spin up the server, and
it boots extremely fast. So, in under a second,
you can hit this server and things are just
working.
So, the thing that I like a lot about
middleman is that it's very easy to come into
middleman from Rails. I said earlier that this uses
tilt, so you can bring all the same gems
and really the same toolset that you hopefully have
come to love inside of Rails into middleman and
start using it without too much of a problem.
So the other interesting thing about middleman is that
it's multi-environment aware. This is, I pulled this from
a default middleman configuration from the boilerplate and simplified
it a bit. But here you, you get an
idea that in your development environment, you can put
extensions in there and activate them. Like live reload.
And you can pull these extensions off of that
directory that is on the middleman website. And then
your production environment in middleman is called your build
environment.
So, in there, you're going to activate certain extensions,
like minifying css, JavaScript, having an asset hash so
that you can basically hash your assets and do
what you do in Rails where you, you give
your asset path a, a hash and a path,
so that you can cache it.
So, once you have this configuration setup and your
webpages and stuff are, are ready to go, you
have to deploy it somehow. And this is another
area where middleman really shines is that, deployments are
incredibly easy. It's two steps easy. There is the
build command, which takes all of the templates and
files and whatnot, and it basically compiles a bunch
of html, css, and JavaScript files into a build
directory. And then the, the best part about it
is that getting that build directory up to your
web server is as simple as rsync. Or, if
you want to throwback to the days of using
DreamWeaver, you could use FTP to upload it. Or
SFTP if you're into the whole security thing.
I developed a, a gem called shart, appropriately, hoping
that it would make for an awkward IT conversation
in some corporate environment, where you can deploy your
middleman website up to an S3 bucket, and you
can actually set the headers that are going to
emitted from that S3 bucket. Mostly caching headers on,
on certain assets.
So there's a ton of tools you can use,
and there's a ton of deployment targets that you
can choose from for middleman. If you do go
the route of deploying to S3, it's an incredibly
cheap way to really run your website. You just
pay a couple cents a month for a, a
personal website. And it's also incredibly scalable, whenever you're
deploying to an S3 endpoint.
So. Yippie. You have a static website. You know,
why should you care about middleman? How is this
going to, to help you scale your application?
Before I dive into that, it's really important to
kind of understand the place that middleman will have
in this spectrum of dynacism, is what I call
it. And it turns out dynacism is not actually
a word, so. I had to ignore the, the
spellchecker on this thing.
And it looks something like this. So the graph
that you saw earlier was actually a static web
application. And if you access PollIt dot com slash
brad on your smart phone, that was another static
web application.
So, that's a very dynamic application, though, in that
the chart has to move and animate and pull
this data from the server in real time. In
under, you know, a second, it has to get
from all of your phones, whether it comes from
SMS or your smart phone, we have to get
it up on that graph in a really short
amount of time.
So that is arguably a highly dynamic application. And
we also have to make things really smooth and
seem very fluid, because that's just what you have
to do for presentations like this. So as we
go down the dynacism spectrum, you have your GUI-oriented
applications.
So, Google spreadsheets - you're probably very familiar with
that. Who hasn't used it? You're doing a lot
of actions, a lot of very short actions, and
you don't want to build applications where you click
a button and then you have to wait for
a response to come back to a server. The,
basically the functions of the GUI are very tightly-coupled
with what comes out of that.
So we kind of start moving down into document-oriented
web apps, like an invoicing application or Base Camp
where you can, you start to get into Rails
territory, where Rails is, it's kind of the sweet
spot between these highly dynamic applications or these highly
static applications.
So moving down into the blogging platform world, you
may have something like Subtle or PostHaven, that is
backed by mySQL, but for the most part it's
just storing these documents in a database server, just
for the convenience of administering many users in a
multi-tendency environment.
And then you go on down to your personal
blog, and maybe some of you have a GitHub
site. Maybe you're using Jekyll for that. Maybe you're
using middleman. All the way down to informational websites,
like a mom and pop shop or Steve's plumping
services.
If somebody wants to make a killing, go out
there and sell these restaurants that have flash websites,
sell them on a middleman website so that we
can get some html out there and have less
flash on the internet.
So, where does middleman kind of fit into all
of this? If you're working on a green-filled application,
it's pretty easy to get started right away. Kind
of what you hear about doing today is, you
build your single page html application over here and
you build your smaller kind of API off to
the side using Sinatra or some type of, of
microframework.
But, since we've been around as a company for,
well, since 2008, we actually started out using Rails
1 point 2, I believe. It was before REST
was even a thing in Rails. So we had
this green-filled application. We spin this thing up. It
was great. We were able to move pretty fast
inside of this framework. And we just set up
this application at polleverywhere dot com. The real time
components we had, I don't know if anybody remembers
RJS, but that was basically how we would make
things appear in, in real time on Rails, was
through these kind of, in retrospect, horrendous helpers that
would emit JavaScript from the server and run on
your web page.
So, we got a lot of mileage out of
that. But it wasn't, it wasn't really enough. We
needed something that was more visually appealing than just
updating some numbers on a table. So that's when
we turned to flex, because they happened to have
this bar chart library that everything updated in real
time. There was just a lot of stuff that
worked out of the box.
And it also just happened, out of sheer dumb
luck, that flash was installed on 99-some-odd-percent of, just
all machines out on the internet. But even more
compelling is that you could embed a flash asset
into a PowerPoint slide because of good-old ActiveX.
So we did this kind of weird hack where
we would embed these swifts that we would generate.
These Flex applications, and we would actually embed them
into a PowerPoint slide and send those out up
on the server so that people could download these
polls embedded right into PowerPoint slides. So whenever they
opened it, nothing really had to be loaded up
over the conference wifi connections in their slide. The
swift application would just connect out to our Rails
app and start pulling data out of it.
So we initially had this flex application in the
same code repository as we did our Rails application.
So as we started to build more customers, things
became more complex, and we were able to afford
a contractor who came in and just started shredding
it on this Flex app and really making a
lot of progress. To the point where we started
getting annoyed at just how many commit messages that
he would have in our, I believe it was
actually SVN at the time. So we broke that
out into a, a separate application. And we were
able to move a lot quicker, in the fact
that we got to separate deployments.
So, we had this contractor working over here on
the Flex app, and he was able to deploy
those Swift assets out to production. And meanwhile the
Rails app team was able to do their work
and, kind of, push their updates out through a
separate release cycle. And, of course, at the time,
it was a lot easier for Flex to work
with XML, so we had a very fashionable XML
API.
So, time went on, and mobile started to change.
It wasn't just about SMS anymore. So we had
to think about the smart phone thing that was,
that was taking off. Really it was the advent
of the iPhone.
So, having a lot of really good luck in
the past with frameworks, including Rails and, and some
of the visual components that Flex gave us, naturally
we gravitated towards using jQuery mobile. And the other
thing, dot mobe extension, was this thing that I
saw on RailsCasts about this new-fangled way to kind
of say, hey, this dot mobe format is gonna
serve up this, this mobile stuff that's kind of
like html.
So, it turned out that was a really bad
idea. The way that jQuery mobile worked got us
about 80% of the way there, but the other
20% was just extraordinarily painful. The jQuery mobile framework
was extremely opinionated. It wanted data in a certain
way. It wanted the DOM to be structured a
certain way. It felt kind of clunky and we
just felt like we were kind of stuck inside
of the, the jQuery mobile world.
So, our team had to, had to do some
soul searching on this front. And what we decided,
at the time, was, you know, instead of trying
to pick a framework, let's actually focus at the
specific problems that we're trying to solve, and let's
focus on picking libraries.
So, that made our thinking a lot more clear,
and we were able to pick exactly what we
needed it. Or, yeah, pick exactly what we need,
and bring it in our application where we needed
it.
So, at the time, as well, there was still
a lot going on in the JavaScript MVC world.
You had sproutcore was one, was one of the
frameworks that we looked at. And that was not
really that good-looking in terms of where we came
from with jQuery mobile.
So, we decided to go full speed ahead and
use backbone.js, which was, I think, at version point
eight at the time. And we knew that we
would probably be swapping out libraries, because the space
was still maturing very rapidly, and there were a
lot of changes. So this approach really let us
hand-pick the libraries we need and swap them out
as something else kind of pulled ahead of, you
know, another library.
So we ended up with this single page mobile
application that we built entirely inside of middleman. So
middleman was handling all of the assets that, that
this application was using. And it uses kind of
the standard sprockets pipeline that, that you have in
Rails. So whenever you build this thing, we have
three files. We upload it to our EngineX server,
and we were pretty happy with how this stuff
ends up working out on production, especially for deployments.
So, if you go down this route of building
these single-page web applications, one thing that you need
to be very aware of is CORS. So whenever
you went to the pollit dot com single-page site
on your phone, that was actually making a AJAX
request to our polleverywhere dot com host to an
API endpoint on there.
So in order to do that without CORS is
you get all these cross-domain errors for AJAX. But
what CORS lets you do is, the pollit dot
com host says, hey, polleverywhere dot com, I'm from
a different domain. I want to make these types
of HTTP requests. A GET request, a POST request,
and, oh, by the way, I want access to
these headers. So our polleverywhere dot com service says,
OK, you're on our whitelist, polleverywhere dot com, so
here you go. Here's the data. Great. You want
to POST something. I'll accept that request. And if
another domain tried to access it, it would just
give them the cross-domain error.
So that's how we, we got around that issue.
And then another really great thing about the way
that we deploy this mobile app is, it's really
easy to deploy this to a CDN. You actually
have these assets that you can push out to
other servers. So you can't really do that with
a Rails app, per se. You can't just take
it and stick it over here on this file
system on this server without having to boot a
bunch of stuff and go through all that.
So another interesting kind of feature of that is
that you can deploy to floppy discs. So when's
the last time that anybody's seen one of these?
And we have these readers here. You pick these
up for about fifteen bucks off of Amazon. And
what I'm actually going to do is a live
hardware demo. Possibly the only one at RailsConf. And
let's plug this in. The really surprising thing, to
me, was that Mac OS actually recognizes floppy drives.
So let's plug this guy in here. Op.
Devices. I think that's devices. Oh, I got to
put the disc in. All right. So you can
hear the, hear the disc spinning. Oh, let's see.
There it is. Boy, that's hard to see up
here. All right. So here's the application. We'll just
launch this.
[laughter]
So hopefully your network connection is faster than a,
a floppy disc, but if you have a customer
living somewhere remote and the only way to get
something to them is by Pony Express, you can
just put this floppy disc in a satchel and,
and send the pony on its way.
The really fun thing is opening network inspector on
this and seeing how Chrome measures this. Is it
latency, or is the file really taking that long
to load?
So, here you go. You just saw a web
application booted from a floppy disc. I can go
to my pollev page and submit a vote. So,
we joke about coming up with a floppy.js library,
because if you can see the inspector down there,
I don't know if it shows, but the retina
assets aren't loaded. Those were too big to fit
on this floppy disc, so we came up with
some fun ideas to span our JavaScript across floppy
discs. But we had better things to do like
fix bugs in production.
So, great. You can put this on floppy discs.
But I think more importantly there is, you can
put them inside of phone gap or Cordova apps
or we have some customers that they want to
bundle our voting application with one of their mobile
applications. So we can say, here is our html
assets, you can put them within your application, and
then whenever there's three-hundred people sitting in a conference
room, it won't overload the wifi.
So, caching aside, there's, there's another component to this.
Flex, Flex started getting old. It started getting outdated.
The writing was on the wall that this stuff
was going to die. So, you know, we set
out to write a utilization app, it was very
natural for us to think, oh, you know what,
let's use middleman. We want this thing to work
on tablets, iPhones, and everywhere, so naturally we're going
to use - oh, what do we call it
today? HTML5. Let's use that.
Let's use that HTML5 thing to talk to our
JSON API. And that worked great. We got this
application out there. It's actually what you saw today,
it's that HTML5 application. We have all the benefits
of caching and the CDN and whatnot. But, but
something came up. Whenever we were looking at these
visualizations day in and day out, we just said,
jeez, these feel really slow, because we were using
short polling. We were hitting our server once every
couple of seconds to get the new data from
our server to, to update on the graph.
So we decided we wanted to do better than
that. We rolled out a Stream API. We actually
wrote our own server at the time, because socket
io wasn't quite what we wanted. I, I actually
gave a talk in 2012 about this, about streaming
resources. So we threw up that streaming API server,
but there were some problems with it.
Back in 2012, when I gave that talk, we
were using AMQP on the backend of this thing.
And there were just a lot of stability issues
with that. It wasn't quite working with the, with
the grain of the web and how resources work
and, and how caching and all that stuff works.
So, we had these stability issues that we were
trying to deal with, and what was really nice
about having these client-side applications, is we were able
to spit up our stream server on its own
host, its own completely different host, and isolate it.
Our team had a lot of learning to do
to understand how to, not only build these real
time web applications, how to operate them. How to
scale them.
So whenever we rolled this thing out, we'd have
crashes, and our client-side application was able to seamlessly,
basically fail over to HTTP short polling. So over
time, our team got much better at just kind
of managing all these pieces. And we had client-side
SOA going on there. So, you can also, with
CORS, if there's several APIs that you have out
there, you can consume those from your JavaScript and
kind of munch all that stuff together client side
and just do whatever it is that you please
with that data.
So, we had so much success with all these
middleman, these single-page middleman applications, that we started to
build all of our other applications in these. So
our approach towards native integrations has been, basically, let's
build a special web browser that has these certain
hooks into JavaScript so that our web developers can,
can be more productive and interact more with basically
the native application.
So we can control a lot of different things
with these integrations from JavaScript. So we start having
all these backbone applications pop up. Now, if you've
dealt with several applications, you may be thinking, jeez,
you know, you're probably repeating yourself with a lot
of different things. So how do you get a
handle on this stuff in this world of sprockets
and middleman?
So what we did is we took all of
kind of the common components of this, the session
components, the models and backbone. We pulled all these
into this one asset gem that then everything consumes
from there. So to make those gems, it's just
like making any other gem. You just say bundler
gem and then whatever the name is of your
gem. Here we have pollitassets. And the kind of
different thing about this gem is that, you check
is sprockets is there. And if it is, then
you start appending all these paths toward the assets
where your asset gem lives.
So, for example, in our lib/assets JavaScripts gem, we
have a user backbone model. We have a poll
backbone model. The stylesheets. We actually use font icons
everywhere, so that we can fit all this stuff
on a floppy drive.
So, we can have all these assets located here,
which whenever all the other applications consume that, they
can have a consistent look and feel that uses
kind of the visual language that we want to
use throughout all of our applications and all the
different platforms.
And of course you have your vendor JavaScript assets.
So if you have four different projects out there,
you're probably using four different versions of jQuery. This
lets us use one version of jQuery and one
version of backbone.
The way that we manage these in our development
environment is just through bundler. So you can imagine
if you have one version of jQuery, well what
happens whenever you bump from the one dot x
to two dot x? You're probably gonna break a
lot of stuff.
But we actually don't have that problem, because like
with RubyGems, you don't really care if there's an
upgrade happening. What you care about is that you're
getting the version that you asked for in your
gem bundle. So we're able to control that by
pushing our pollev assets up to a basic Git
repository, and we reference that from this gemfile. You
don't actually see the, the git reference in there,
but you can see that, in this case, we're
saying, hey, I want to use the new feature
branch of pollev assets. The assets path thing above
that is a nice little hack so that if
you're making changes to the pollev assets project, you
can pull those locally so that you don't have
to run bundle update every time.
And what's cool about middleman is, if you're developing
these pollev assets locally, whenever you reload middleman, it
actually picks up the changes from the, the assets
gem. You don't have to reboot the server or
do anything crazy like that.
So, the way that we can build new features
now, let's say that, let's say the worst case
scenario, I have to build a new API to
support a new feature. I can branch my Rails
app project, and I can say, hey, branch this
off. It's called new feature. I'm gonna add some
kind of new visualization to it that needs a
new API. So I can build that API out
on my server. I can develop that locally. And
I can point my middleman project at my local
server.
And I can branch it in here and I
can branch pollev assets, and basically have three different
branches or, sorry, one branch and three different repos,
all working on the same feature. And then whenever
I go to deploy, obviously deploy the API server
functionality first, and then it would go on to
roll out these middleman single page websites.
So, does it work? I think so. It's, it's
worked really well for us. We have to deal
with some weird kind of bandwidth constrained environments, where
you can't trust conference wifi connections. And it also,
I realized that it worked really well for us
when, about two months ago, Microsoft wanted to launch
their PowerPoint 2013 store inside of the Microsoft Office
Applications. They actually have an app store inside of
there. And we were able to pull a bunch
of assets from our mobile application into our sprockets
asset gem and reuse all of that stuff, most
of it, in the PowerPoint 2013 app. And then
we were able to quickly make some customizations to
that whole user experience to make it fit in
with Office 2013 a little bit better.
And, of course, whenever you're running a single-page JavaScript
application inside of this little web iframe, essentially in
Office, it feels very close. It feels very native.
So that, that worked pretty well for us. But
I think, even better than that, than reusing functionality
is, inevitably, whenever you work on some of these
projects, you make some kind of improvement. Something just
feels better about maybe handling a login or some
kind of status code or something.
So these improvements that we make in these individual
projects, we're able to pull them back into the
sprockets assets gem and then push them back out
to all of our other projects. So all these
other projects you see up here benefit from the
PowerPoint 2013 app, and that just keep, kind of
reinforces itself. Reinforcing itself.
So, in a sense, the sprockets assets gem turns
into this little, perfect little framework for your organization
that's extracted in the right way, where it's being
extracted from stuff that's actually being used and being
proven as successful by customers.
So, that's a pretty good overview of one of
the more complex middleman deployments I would say is
out there, and just managing kind of multiple projects.
One of the downsides and also plus sides of
middleman is that there's no out-of-the-box JavaScript MVC app
solution in there. So in the case of backbone,
it's up to you to organize all the assets
in some way that, that makes sense. And different
JavaScript frameworks have, some are more organized than others.
So that's both a blessing and a curse.
It worked out for us because there really wasn't
a framework. We kind of came up with our
own and we didn't have to deal with, with
somebody else's bad framework.
So, that was the overview of the highly dynamic
website, which has all these static
applications stashed everywhere.
What about static websites? The other side of the
kind of token where Rails fills the middle ground?
So, we're developing a lot of content for, for
our website. There's an explosion of use cases and
all sorts of stuff that we have to implement.
So one thing that we'll probably do on this
front is extract out a content directory from Rails
app. How many of you have a, in your
Rails app, it starts out. You have kind of
the home page, and then it turns into this
directory called content, and then you have all these
content pages. Maybe you end up with twenty or
thirty of them one day and, before you notice
this big junk drawer of content, and you have
to look in your routes file and kind of
make sense of all this stuff.
So, in middleman, you don't, the directory structure is
the routing structure. So that stuff checks out a
lot nicer. And it's just much easier to handle
this content. And there's also much better support in
there for different things like image compression. You can
png crush everything. So if you have designers building
image assets and they don't understand the technicalities of
making pngs much smaller, middleman can take care of
that in its asset pipeline.
And, of course, on a static website like that,
you still have some dynamic components. You can't just
throw a static website out there and just tell
everybody, like, oh, no, forget about login state. There's
some common things that people want to see, so
you can take care of that with JavaScript, and
you can run all that stuff client-side.
So you can have some really lightweight JS that
checks with the server. Maybe it checks for the
presence of a cookie to see if, if the
user's logged in or not. And then, of course,
if you have a contact us form or anything
like that, you would test these integration points with
Rails so that whenever somebody types in stuff into
a form and submits it, it hits your Rails
app and hopefully it doesn't blow up.
And, of course, you have other JavaScript applications, like
Google Analytics, Optimize the Stripe.
All these little JavaScript-y
tools that you can throw in there. And, you
know, it makes it a little more dynamic.
So the nice thing about a static website is
that you can't get taken down that easily, especially
if you have this stuff up in an S3
bucket or stashed in CDNs. I was at a
middleman meet up about a year ago, and the
folks from nest dot com were there. And they
said they were using some CNS or some, some
dynamic backend, and they kept being mentioned in the
press. And their website kept getting really slow.
So they looked at a few things and I
think one option involved just throwing hardware at the
problem. And they decided that wasn't sane. Let's just
build a static website with middleman and push this
stuff out there on a really simple server. And
they no longer had any of these problems where
their site would get taken down from a, an
influx of, of traffic.
So there's a lot of things to think about
whenever you're building these middleman applications.
I could probably
give another two or three talks just on these
items alone. If you want, ask me questions about
this stuff later. But I don't have time to
cover that now.
And, of course, I encourage you to get out
there. Build your next website with middleman, even if
it's a personal website or a blog or something.
Try it out. If you're working on single web
page applications, you could take your handcrafted stuff and
throw it into middleman and start using that. Start
using the asset pipeline.
A lot of these ideas that I've talked about
today, and some of the things I didn't talk
about, our company polleverything is
actually abstracting out this
framework and we're going to be releasing these bits
into gitfannypack, or fannypack is gonna be the name
of this, this kind of framework that uses middleman
and deals with stuff like testing, binding to different
environments and all that stuff.
And, of course, if you want to work on
this stuff every day and get paid for it,
you can join at team at polleverywhere. So, that's
my talk. I'm Brad Gessler. I'll be posting slides,
links, and, and code, up on Twitter whenever I'm
not working on these slides. Thanks.