The Uselessness of Amazon EC2 Instance System Log

Just a thought about AWS WebUI drawbacks. 

How is that possible that there’s no indication on what’s going on with an AWS EC2 Windows server when it is restarted? I’m talking about Instance System Log. It’s completely useless.

The Case: Manual Windows Updates

It’s just a nightmare when you need to do manual Windows updates. There’re cases where Windows servers simply cannot afford to have automatic Windows updates enabled. I’m not going to go into any specific details here.

You normally install Windows updates during the agreed maintenance window, an they ask you to restart the server. OK, no problems, go for it.

But once you click the “Restart” button, that’s it. Light’s out. You have no information on what’s going on with your server. Perhaps the server restarted, and now says “Please do not power off your machine. Installing update 1 of 172 …” But you have no way to see that. System Log is empty, pretty much as my pockets on a Saturday night.

How is that even possible that you don’t get such notifications on AWS WebUI? Server may be inaccessible for 20 minutes, and you have no idea of what is happening. Is it still applying updates? Or has it restarted again after installing them?

Having waited for 20 minutes, should you stop the server and start again? Or should you wait for 20 more minutes? If it’s still applying and/or configuring updates and you manually force it to stop, will it crash and refuse to boot afterwards?

Perhaps it has already crashed and you should start recreating the volume from a snapshot? You don’t know.

AWS CLI is as bad as WebUI in this case:

$ aws ec2 --output=text get-console-output --instance-id i-****b827
i-****b827 3/21/2014 10:22:08 AM: Waiting for meta-data accessibility...
3/21/2014 10:22:09 AM: Meta-data is now available.
3/21/2014 10:22:14 AM: Message: Windows is Ready to use
3/21/2014 19:49:03 PM: Waiting for meta-data accessibility...
3/21/2014 19:49:06 PM: Meta-data is now available.
3/21/2014 19:49:10 PM: Message: Windows is Ready to use

This is all you get between the 2 server restarts.

AWS CLI reference says that for Windows instances, the instance console output displays the last three system event log errors. This seems useless.