Amazon EC2 Instance inaccessible because of s3fs

Hi again!

Long time since my last post but always wanted to continue this blog - as it turns out some might be actually reading it ;)

During the time I was gone, I changed jobs a couple of times, even changed countries. I am now based in London, UK - a rather exciting city. Now working a lot with Amazon AWS and Java EE.

This blog post is about problems we had setting up auto scaling on Amazon AWS EC2 cloud. We were using a GUI front-end named Asgard (the home of the gods!) by Netflix trying to set up clusters and test them.

As we are moving more and more of our infrastructure into EC2, some problems become more and more apparent, especially the ability (or in our case - inability) of applications to work in clusters. There are some things that need to be considered, before creating a cluster of you application, which are not in the scope of this post.

Long story short, we needed to make some short cuts and instead of using Amazon S3 as it should - via its HTTP API, we went the s3fs way... it is a neat helper tool but can have nasty effects like memory leaks (saw it grab 50% of the machines RAM) and wild CPU usage... as it had in our case.

I was during a jMeter performance test of our two moved clusters, one is a PHP frontend app cluster and the other is a Perl service application cluster. Both of them started of as small instance clusters however the tests revealed that our Perl apps have a rather annoying problem with going really well and then just dying when overwhelmed. So we thought to go with medium instance sizes for the Perl cluster...

The medium instances hover did not start properly, they couldn't be ssh'd and the "get system log" option revealed that the setup didn't finish. The problematic thing about this was, that even though the small instances started without issue, the medium or large (we didn't try higher ones as it didn't seam there is a point) won't start. In the AWS console the instances passed all checks (which seam like a bit of shitty tests IMHO) but still there was zero movement.

After a few hours of trying different options to connect to the instances, we actually paid for Amazon's business level support - which is amazing. It costs you about 10% of your monthly fee or $100 whichever is higher, but assuming we would spend a week on figuring this out... well worth it. After about 30 minutes with one of their support staff, we finally found the problem.

We had a s3fs share mounted straight to /mnt and with no __netdev options (which basically means, that it need the network connection in otder to be mounted). After changing the mount point to some other folder and adding the fstab option, everything went back to normal and the instances

What we assumed is that the small instance was slow enough to start s3fs after it had started networking, therefore finishing the boot sequence correctly.

Hope this helps anyone stuck with the same problem. If an instance is not booting properly - look at fstab and use s3fs as a temporary option.

Cheers,
P

Symfony Dependency Injection - a really cool PHP tool

Hi!

Today I want to talk a little about a great design pattern called Dependency Injection and it's implementation by SensioLabs called simply Dependency Injection Component.

Now dependency injection is basically the inversion of control in you classes. It can help you to create decoupled structures that can be configured at run time which gives you a lot of flexibility. Let's take a look at an example:

Let's presume, you need to user a tool class inside some other class:

class User {


private $tool;


public function __construct(Tool $tool){
   $this->tool = $tool;
}


public function doStuff()   
   $this->tool->doSomeStuff();
}


As you can see, I've user the strategy pattern, so every time I want the USer to ->doStuff() it uses the ->doSomeStuff() method of the Tool class object.

Now this way is pretty neat, because it avoids  hard-coding the Tool into the User allowing for polymorphism. However creating a lot of such dependencies can be a pain if you have some more complicated stuff to do. Additionally there can be some problems with instantiating the Tool class before usage.

Symfony dependency injection can solve both those problems as it decouples elements and creates them upon usage, not upon configuration and helps to easily create complicated dependencies just before usage. Let's look a the example ripped out of my ZF project:


//retrieving mailer options from configuration

$mailerOptions = $this->getOption('mailer');


//we create a builder object, that will later build our classes
$serviceContainerBuilder = new sfServiceContainerBuilder();


//regitering the mailer service
//this means, that when we call the mailer service
//the Zend_Mail class has to be instantiated with parameters
//and a function called with some additional params


$serviceContainerBuilder->register('mailer', 'Zend_Mail')
->addMethodCall('setDefaultFrom', array($mailerOptions['from'], $mailerOptions['from_name']))
->addMethodCall('setDefaultReplyTo', array($mailerOptions['replyto'], $mailerOptions['replyto_name']))
->addMethodCall('setDefaultTransport', array('%mailer.transport%'));


//everything should be pretty straight forward
//I'm creating a builder instance and configuring it
//with the object class I wan't to have created
//and it's parameters
//the setDefaultTransport param is still open, when you add
//params like %param% sfDI knows that you want to set the param
//before service class instantiation


//setting to Zend_Registry to get the serviceContainer whenever I want
Zend_Registry::set('serviceContainer', $serviceContainerBuilder);



Some of you, who have some experience with dealing with Zend_Mail, know that you need to have a mailer transport of type Zend_Mail_Transport_Abstract.

In my application I have two bootstrapping (preparing basically) files:
1) the main application bootstrap file (from which it was I took the above example)
2) the PHPUnit bootstrap

The testing reasons I want to have a different transport in the second bootstrap that in the reall one. The first mailer should normally send the email but the second one should just log it in a file. Here the dependency injection copes in really handy:

1) Main bootstrap:



$serviceContainerBuilder = Zend_Registry::get('serviceContainer');
$mailerOptions = $this->getOption('mailer');


//I want the main transport to be a posrmarkapp.com using transport
$serviceContainerBuilder->register('mail.transport', 'FT_Mail_Transport_Postmark')
->addArgument($mailerOptions['transport']['api_key'])
->setShared(false);


//the setShared(false) simply tells the container that the mail.transport object isn't unique


//here is the interesting part
//I tell the Container to inject the newly configured container, as the 
//"mailer.transport" argument to the mail.transport container


$serviceContainerBuilder->setParameter('mailer.transport', new sfServiceReference('mail.transport'));



2) The second bootstrap (PHPUnit):



$serviceContainerBuilder = Zend_Registry::get('serviceContainer');
$serviceContainerBuilder->register('mail.transport', 'Zend_Mail_Transport_File')
->addArgument(array('path' => APPLICATION_PATH . '/..tests/log'))
->setShared(false);



This example shows just how you can create another transport for another purpose, just as easily.
Now we want to see it all in action:


//in some form processing command, we retrieve the container and create the service

$mailerService = \Zend_Registry::get('serviceContainer')->getService('mailer');


$tpl = //obviously we need some template


$mailerService->setBodyHtml($tpl->render());
$mailerService->addTo($emailTo);
$mailerService->send();



It's that simple, we can additionally set some parameters before getting the service, adding different behavior to the container.
Some of you might say that you can preconfigure the Zend_Mail class instance and set if directly into the registry. True, but remember that not all pages result in the usage of the mailer, so you would probably waste a lot of resources holding a lot if instantiated objects in the registry.

Hope you enjoyed the examples and start building your complicated class relations with the Symfony Dependency Injection Component. For more information go to http://components.symfony-project.org/dependency-injection/

cheers,
Peter

SilverStripe CMS for PHP programmers, not for dummies

Hi there!

Recently I've came across a pretty standard problem:

1. I had a simple php website with little php really, some routing, contact form handling, teamplating, nothing fancy
2. I hate doing boring stuff like CRUD's and CMS's
3. I make a lot of changes to the website
4. The website comes in 2 languages

If you think "just user some cms" - well I had the same idea. I started from the best known to me - Wordpress. It's pretty robust, but not being written as a CMS, but as a blog, it didn't really fit in. Of course I could force WP to do what I wish, but anyone who developed something in WP knows it's not really for programmers (or by programmers IMO).

The second choice was on pretty standard CMS's like Joomla or  Drupal, still programming was really problematic, extending stuff hard, since I wanted pretty much nothing form Drupal, and had to leave all that was on the old page.

Finally a friend of mine introduced SilverStripe, a New Zeland made CMS/Framework.
I'm a sceptic by nature but this I loved at first sight.

1. You program first what you want (pretty much no prerequisites), and let the framework build the rest. It's really for programmers, you can't just click away compromising modularity or re-usability. Everything is a class, that you can create and use over and over.

2. Every one of my custom objects (like for example ProjectRealisations) are encapsulated in a single class, that was pretty easy to set up and extend.

3. Multilingual setup for the CMS (options to get any additional language) - 5 minutes.

So what I've done is set up a completely custom website with a CMS for multilingual content (with ACL cf course) in a few days, while learning the CMS/Framework (ssbits.com - a big help).

I strongly recommend the framework for small websites, for lazy programmers (like myself;) )

See you next time,
Peter