Wednesday, June 18, 2014

Pull Requests instead of Emailing Code



If you modify code from an open source repository, such as GitHub or BitBucket, here are the reasons why you should submit a pull request to get your code back into the main repo.



  • To avoid paying the stupid tax.
    • In short, your feature or fix will be available in future versions that you might want to use. If your code doesn't get into the master branch then it makes it difficult or impossible to keep up with future versions.
  • Make open source better.
    • If you've fixed a bug or added a feature then it's highly likely that others will need that.
  • Improve your resume.
    • More and more recruiting managers are looking at what you've done when they're hiring you. Having contributed to an open source project is a great way to show that you've done real work that people are using.
  • Get Credit
    • You've done the work. Now get your name on that repo and take credit for some of the work that you've done.
  • Use the Tools
    • Do you use Git? Have you ever submitted a pull request? These are questions you might be asked at an interview. Try it and practice it so you can demonstrate this skill. Even if I was hiring a junior right out of school I'd expect them to have at least done this.
  • Understand someone else's code-base
    •  Making a contribution to someone else's code-base forces you to read their code and understand their style and way of architecting a project. This is an invaluable skill as 80% of the work you do is reading over code. Even if you wrote it you'll have forgotten it to the extent that it becomes foreign in a few months time.
 This list came from a conversation that I was having with the owner of the ABot project on GitHub. ABot is a web crawler (spider) built for speed and flexibility.


He was telling me how other developers will add features to ABot and then email the code to him. If he decides to accept the code and integrate it into the project then it's all getting committed by him under his name. He was telling me how he doesn't want the credit for this work. By not submitting your code through a pull request on GitHub it makes it very difficult to give the credit to the person who did the work.

Tuesday, June 17, 2014

HTTP Status Check



I recently created a Node.js utility that will cycle through a list of URLs and check their HTTP statuses:

Npm: http-status-check

GitHub: http-status-check

This utility came out of a need to keep daily tabs on a number of URLs and the statuses that they were returning.

I plan on expanding it with other input and output adapters. Obvious input adapters would be databases. What else can you think of? Obvious output adapters would be email and any other type of messaging system.

Pull requests are welcome.

Update 4/July/2014

Added an excludedHeaders option. This is a list of headers that you want the check to fail on if they are present in the response from the server.

At first blush this may seem like a strange check to make. The common use case is the X-Powered-By header. This header allows the server to advertise the technology that is powering it. As a security concern, when possible, I'll remove this from sites that I publish. I feel that telling malicious attack bots what you're running on will help them exploit any vulnerabilities your stack might have.

Update 5/July/2014

Added an expectedText option. This is text that we expect to be present in the body of the response from the server. By default it is case insensitive but you can change that by supplying an object instead of a string.







Saturday, June 7, 2014

Using WebSockets when your Reverse Proxy doesn't allow it

As developers we don't always get to choose where our software runs. We often face economic or other restrictions based on infrastructure that already exists.

I recently moved a Node.js application from Linux server to a Windows Server 2008R2. Crazy right? It's working surprisingly well in the Windows environment. IIS 7.5 already owned port 80 so I had to setup a site on IIS, bind the domain to that site and use it as a reverse proxy to the Node app which was running on an arbitrary port.

In this case it happened to be IIS in any other case it might be Ngnix, Apache or any other server or reverse proxy that is between your Node.js application and the web. The problem that I faced is that this version of IIS does not support WebSockets so it looked like I couldn't use that and had to allow socket.io to fall back to long polling for this application.

There is, however, a solution, and it's rather simple.

Your site's facade, let's call it mdomain.com, is running on port 80 on IIS which is configured to run as a reverse proxy passing all traffic through to port (say) 4444 where your Node application is running.

When a client (a browser) connects to your site you provide it with the usual payload of HTML, CSS and JavaScript and in that you also provide it with the port number or sufficient information for the WebSocket part of the client to make a direct connection to your Node.js server and bypass the reverse proxy completely.

Using this little trick our site can remain on the default port going through the reverse proxy and all our WebSocket traffic can run over the application specific port.

Tuesday, June 3, 2014

JavaScript Dependency Injection.

Using JavaScript Dependency Injection for Spies, stubs and mocks in unit testing.

I was in the process of picking a Dependency Injector for use in a project in JavaScript. My default to narrow down the top three is to use Google as my initial search and then GitHub's star count as the top-three filter and then compare the features that I need.

After looking a SinonJS, sandboxed-module, rewire, and proxyquire I decided to be slightly more analytical and objective about how I compared them. I decided to include the Watch and Fork count in addition to the Star count. I created a spreadsheet that compared them:
 

watch star fork Avg
rewire 20 425 16 0.48
sinon 52 1471 233 2.08
sandboxed 10 190 24 0.29
proxyquire 2 195 12 0.15

84 2281 285 3

I averaged the modules giving each of the attributes a one third weighting. Given that starring is 27 times more common than watching you might question this weighting. I justify it because starring is a less costly action (no update emails) and more likely to do than watching.

Seeing this as a pie chart makes this information easier to absorb. The clear winner amongst these four modules is SinonJS with almost three-quarters of the attributes.







Thursday, May 8, 2014

Thinking outside the Twitter Bootstrap (monitor) box

I'm doing a whole bunch of development using Twitter's CSS Bootstrap framework and while resizing the width of the browser I'm never sure when the grids should change based on the row and col-xs-X, col-sm-N, col-md-N, col-lg-N classes.

Then I had an idea. I'll pin the left side of the browser to the extreme left of the monitor and physically mark off on the top of the monitor where each width size starts.

To find the points on your browser where the switch takes place do this use the Bootstrap's Gid Template page and after pinning the browser to the left of the monitor size the right edge of the browser and you'll see the grids jump as they change and this will allow you to mark the changes. You'll need three stickers for these as you're dividing up four areas. I borrowed stickers from my kids sticker collection which read "Smart Work", "A Big Effort", and "Super." If you don't need encouragement like this then any sticker will work.

I then took four white adhesive labels and cut them up and labeled them XS, SM, MD and LG and stuck those in the appropriate places between and next to the dividers.

If I want to see the effect in each of the four bootstrap sizes I slide the right side of the browser under the XS, SM, MD and LG labels. If I want to see the "jump" as the size changes I size around the dividers.

Sized at SM

Sized at MD

Friday, April 11, 2014

IIS Reverse Proxy to multiple sites running under a single Node.js instance

Subtitled: How to setup Node.js to handle multiple sites and to run on Windows while IIS still occupies port 80


I did this on a Window 7 machine which was running IIS 7.5

 

Setting up your local box for testing

In your HOSTS file add some sites to test:
127.0.0.1     node1.com
127.0.0.1     node2.com
127.0.0.1     node3.com


IIS

Create a new web site in IIS:
Open IIS.
Right click on Sites.
"Add Web Site..."
Site Name: MyReverseProxy
Physical Path: C:\inetpub\wwwroot (can be anything as we're not going to use it)
Host name: node1.com
Now click on Bindings under Actions on RHS
Click Add.
Host name: node2.com
Click Add.
Host name: node3.com
Close.
You now have 3 host names running on port 80 on IIS and your HOSTS file will direct all local requests to this web site.

Click on MyReverseProxy on left panel. If the URL Rewrite widget is not under IIS then you might need to install it and might also need to add Application Request Routing (ARR). I can't remember if it came installed by default on IIS 7.5 or if I added it.

Open the URL Rewrite widget and click on Add Rule(s)...
Select a Reverse Proxy rule and click OK.
Server name: localhost:3000/ (or where the Node server is running.)
If you're using relative URLs in the Node.js code then you don't have to rewrite the domain names of the links in the HTTP responses. If you are using absolute names then you're in trouble if you're handling multiple domains as you won't know at this stage which one it's for. The only solution I have for this is to use relative URLs and DO NOT check the option to "Rewrite the domain names..."

Now start your Node.js application on port 3000 and try and access the sites node1.com, node2.com, and node3.com and they should all return the contents of your site running on port 3000.

We're not done yet. If, in Node.js/Express, you take a look at the req.headers.host value you will see that it reads "localhost:3000" and has not passed through the value you were expecting which was one of node1.com/node2.com/node3.com

To get this to work you need to go to this folder with an Administrator command prompt:
C:\Windows\System32\inetsrv
and run this command:
appcmd.exe set config -section:system.webServer/proxy /preserveHostHeader:"True" /commit:apphost

What this command will do is open this file:
C:\Windows\System32\inetsrv\config\applicationHost.config
and find the section called <system.webServer> and change this:
        <proxy enabled="true" />
 to this:
        <proxy enabled="true" preserveHostHeader="true" />
Recycle the app pool on your reverse proxy site in IIS and try and access it again. The host "head" property will be correctly set.

Now the final "thing" you want to do is to be able to handle multiple sites from your Node.js/Express application.

Here is some basic middleware in app/server.js that will switch between sites:
app.use(function(req, res, next) {
    console.log('Domain is: ' + req.headers.host);
    switch(req.headers.host) {
        case 'node1.com':
            break;
        case 'node2.com':
            break;
        case 'node3.com':
            break;
        default:
            console.log('Unknown domain: ' + req.headers.host);
            break;
    }
    next();
});

You would then setup routes that appropriately handled routes for each host that you were expecting. Where routes are ambiguous you would put a switch statement like above in there to arbitrate among domain functionality.

You could also put a node-proxy in front of these sites to switch between different node apps but then you're defeating the purpose of IIS which can already do that and more efficiently so I can't see the need for a node-proxy.

Some of the answers from this Stackoverflow question might help readers of this topic.

Friday, April 4, 2014

ExpressJS and MongoDB End to End

Short URL for this blog post: http://x.co/dcc2014

This blog post accompanies the presentation called ExpressJS and MongoDB End to End at Desert Code Camp 2014.1.

If you want to follow along with this presentation then there are three items you need:
  1. Install Node.
  2. Install MongoDB.
  3. Clone the ExpressJS Sample.
Once you've cloned the sample this locally you should be able to run it using:
node app.js
If MongoDB is running on the default port then it should work.

Give me feedback when the presentation is over: Guy on Speaker Rater

Resources from presentation: