Don't turn your unit tests into integration tests

It’s a common issue to use tools that aren’t very mock/stub friendly. In these cases it’s easy to fall in the trap of abandoning unit tests for integration ones. This post will analyze the impact and how to overcome this problem.

Rapid feepback loop

TDD and BDD require a rapid feedback loop, you need a suit of tests that will provide you information in a timely manner about the correctness of your code. When an external service is required or even a service running in a docker process, the net latency is introduced into every remote call which will cause a noticeable impact on your suite performance wasting precious time.

Line ruid

The worst con you’ll see is unexpected behavior. In fact this is so bad it may entirely defeat the purpose of testing. Calling a a hot service will make your test suite subject to network outage, dns failures and a myriad of other problems.

Help Mock-wan, you’re my only hope

The interference of external variables on the test suite should be minimal and mocking is the best way to insulate your application. When dealing with tools that don’t support it as first-class citizen, build wrappers to allow mocking and stubbing to take place.

Ruby sniplets for optimized tasks

Reading a file

enum = file.lines
100.times{enum.next} # offset 100 lines
enum.take(100) # take the next 100 
 	

Reading from socket

require "socket"
server = TCPServer.open(8080)
loop do
    Thread.fork(server.accept) do |client| 
        reply = "Connected!"
        client.puts(reply)
        client.close
    end
end

The loop directive is more terse than while true

Dynamic dispatch

This is my solution to the coderbyte’s Reverse polish notation challenge. It illustrates ruby’s send method dispatch

  expression = "+ - 1 2 5"
  operands = []
  evaluated = []
  
  expression.each do |c|
    case c 
    	when /\d/
      		evaluated.push(c.to_f)
        when "-", "/", "*", "+", "**"
      		operands = evaluated.pop(2)
      		evaluated.push(operands.first.send(c, operands[1]))
    end 
  end  
  evaluated.first.to_i

Rails gotcha, avoid active record magic when dealing with bulk objects

Consider you are trying to recover all tags, you’re inclined to write:

tasks = Task.find(:all, :include => :tags)

But tags are often value objects, only their name matters and there’s often no logic attached to this state. So consider using instead:

tags = Tag.select <<-MLINE
      *,
      array(
        select tags.name from tags inner join product_tags on (tags.id = products_tags.tag_id)
        where products_tags.product_id=products.id
      ) as tag_names
    MLINE

It’s much less sexy but much faster.

The DAO Failure

I’ve been keeping up with the explosive news this week about the DAO failure and one comment really put things in persepctive

What strikes me as even more off-putting is the fact that Ethereum developers had money invested in the DAO. Imagine if a Bitcoin company went bankrupt, and then the developers planned a hard fork to get their money back – because they are personally invested in the company! Even though the code was fine and everything executed as it should, they lost out – and it seems odd to permit a hard fork because of it. It’s a dangerous precedent.

Steve Patterson

To those who are not up to date, DAO or Decentralized Anonymous Organization is a smart contract running on the Ethereum network. It’s a piece of code that enables the partners, i.e. the DAO token holders, to vote on investment decisions on where to apply the 150M+ dollars fund raised during the buy-in and democratically decide the fate of the funds. The attacker exploited a bug in the ethereum contract and drained 50M+ usd from the fund. This made some DAO owners call for a hard or soft fork in the ethereum blockchain.

Now the decision faced by the community is to face the consequences of the code in the ledger and it’s execution or leverage the collective weight of the community to steer the network where they believe is morally acceptable.

Using monit to restart processes

Monit is an invaluable tool for configuring an automated production environment, it offers streamlined and simple monitoring, takes actions based on defined conditions and it’s syntax is incredibly easy to read.

The following example is used in the Bseller.com.br production environment to restart misbehaving tomcat processes, monitoring for cpu, process memory, port binding and lastly system memory.

check process tomcat with pidfile /var/run/tomcat5.5.pid
	start program = "/etc/init.d/tomcat5.5 start"
	stop program = "/etc/init.d/tomcat5.5 stop"
	if cpu > 80% for 5 cycles then alert
		if cpu > 90% for 10 cycles then restart
		if totalmem > 1500.0 MB for 5 cycles then restart
	if failed host 127.0.0.1 port 8080 then alert
	if 5 restarts within 5 cycles then timeout
check system localhost
	if memory usage > 95% for 5 cycles then exec "/sbin/shutdown -r now"

For a basic introduction to monit, check the official docs at http://mmonit.com/monit/

Http Request Life Cycle

Between typing www.wikipedia.org in the browser and reading about the Unexpected Spanish Iquisition there’s a lot of magic going on.

DNS lookup

First the computer has to make sense of the url, that is, translate it into a meaningful IP address where it connect and interact. The OS queries the nameserver for correct record and caches it locally so future queries won’t need to hit the remote server.

Estabilishing a connection

With the IP at hand, the OS can estabilish a socket connection between the client machine and the remote server. The stream socket is the base for HTTP communication which is the application level procotol used by websites and is identified by the remote server IP, the service port(in this case the default 80), the local port opened and local network interface IP. These ports are binded and data written to one is relayed to the other.

HTTP kicks in

Request

With the two way connection in place, the browser writes a message to the server:

GET / HTTP/1.1
Host: www.freebsd.org
User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.7.7) Gecko/20050414 Firefox/1.0.3
Accept: text/html
Accept-Language: en-us,en;q=0.5

The first line tells the server the client is using http 1.1 for communication and using the GET verb on the root directory. The client also send several information using HTTP Headers, meta information exchanges by the two peers about the information. For instance, the Accept header tells the server the client knows how to understand HTML and the Accept-Language one says the content should be in english preferably.

Response

The server will then reply with the index.html document sitting the root of the server by writting an HTTP response to the socket. Like the client, the server will respond with several metadata about the content it is streamming. One important point is the status code 200, http have several status codes about server availablity, erros, auth and even if the content replied is up to date or a stale copy.

HTTP/1.1 200 OK
Date: Fri, 13 May 2005 05:51:12 GMT
Server: Apache/1.3.x LaHonda (Unix)
Last-Modified: Fri, 13 May 2005 05:25:02 GMT
Content-Length: 33414
Keep-Alive: timeout=15, max=100
Connection: Keep-Alive
Content-Type: text/html

<!DOCTYPE html>
<html>
<img src="/spanish-inquisition.jpg"/>
</html>

Rendering

The browser reads the html payload and displays it using its rendering engine.