Notes on Amazon EC2 Image Creation Fails for Windows Machines

Disclaimer:

• This is my notes on the topic and not necessarily a complete solution. In fact, probably it contains some very inaccurate information (but you can help to rectify this via your comments.)
• This article also pertains to EBS Backed Windows EC2 running instances.

One of the things all of us should do is that once we have a stable machine running, we'd want to save the image of it so that if, for some reason, we completely trash the running image, we have something to go back to without spending a lot of time putting the machine back from the scratch.

Obviously I have tried this with varying degree of success and information on this is scattered around the Amazon AWS forums and on the web. The most common of the errors would be that the Image creation begins and then after a long while it says "Failed" but without much explanations why. Some people on the net are saying the success rate is 40%. I was actually relieved to find this because I thought it was just me that's suffering this much failure rate!

The Amazon documentation is not very clear to me as to what and how to do these things. It seems to require you have gained some more "under-the-hood" knowledge of this. And as a just a casual user of the system I expect that I should not need to learn everything to use it.

In addition to that as a "basic" user, I don't get any direct support from Amazon. That's at least $100/mo or much more for silver or gold support privilege even in these kind of the situations when the failure (even just to explain what to do) is on their side.

I also highly suggest you install Elasticfox on your browser. Since the following descriptions will be based on how I'd do things on Elasticfox.

My Findings on Creating a Copy of Running Image

• To do this on Elasticfox, go to the Instances list, right click and select Create Images (AMI) item.
• Many people reported that when creating a copy from the running image, it will stop and restart the instance. This does happen. Worse yet, it will be stopped for the entire duration of when it is taking the snapshot, depending, this could be 10s of minutes. You'd probably want to schedule a down time. I should have done this just as soon as I have created the initial instance before went into production. Elasticfox does have a menu item not to reboot the instance I have not yet tried this.
• Some people say it has to be a Running instance, but that seems to be not true. I can image from a stopped instance as well and I have made images successfully this way via Elasticfox. In fact, that has more success rate than doing it on a running instance.

Some of The Issues I Have Discovered That Cause Failures

• Again, copying the running instance will cause a restart, so be prepared for the downtime.
• If you have already stopped the instance, it is at least more likely the imaging will succeed. A few caveats on this are that, at least in my case, the Elastic IP will get disassociated with the instance as shutting down the instance will detach the Elastic IP (wish they did not, at least for a short time), and also the machine's local addresses will change after restarting. This means that if you are locally pointing systems, for example, SQL, then the that address have to be re-configured in each application.
• Processes like SQL or other high I/O or CPU consumer items should be stopped. In fact, you should pare down the services to essentials (like SQL, agents, browser etc.) before taking the image copy. The one exception to this is not to stop the Amazon's EC2 services that came with the original Windows image whichever they exist.
• If you have other volumes mounted that also leads to a failure. Stop all services and applications you don't need to be running then dismount any "drives" and then make the image copy. This seems not be an issue if you are imaging a stopped instance.

Well, good luck and if you find this article correct or wrong, please let me know in the comments so that I can actually improve it.

IIS 6: Installing Duplicate Certificates to Multiple IIS 6 Servers For Server Farm Implementation

Symptom:

You have more than one servers that are hosted through a load-balanced router. The external address is assigned to (obviously) a single Fully Qualified Domain Name (FQDN). You want to install the same server certificate that represents the same domain name across all your IIS 6 based servers.

Procedure:

First off, yes you can do this. The procedure is very clearly explained in this Microsoft article:

http://support.microsoft.com/kb/313299

Just in case we lose above article, here is the gist of how it is done.

• Request and install the cert on the first server as you would normally do. Don't do the request from other servers, if you do, remove any pending cert requests.
• Open the MMC and add the Certificate module.
• Open the Computer Account then Personal certificate folder.
• Navigate down to the Web Certificate you want to export.
• Right click tasks and Export.
• In the wizard select to include the Private Key and include all certificates in the path in the next page.
• Copy the exported result to another server
• Do the same MMC stuff
• Import the stuff you exported into the Personal store
• From the IIS certificate section of the Directory Security, do the "Assign an Existing Certificate"

That's basically all you need to do.

Getting an XmlNode (or XmlElement) Out of XPathNodeIterator (C#)

Symptom:

You have a code you iterate through a group of Xml Nodes (for most people that's XmlElement). You have however noticed that once you get to a specific spot, there is no easy way to perform the complete node operations against the iterator since Casting of the iterator to XmlElement results in a compilation error.

So, you cannot cast the Current of the iterator like

XmlNode node = (XmlNode) i.Current;

(I really think this is a poor design myself since it goes against all our intuition.)

Solution:

Well, there actually is a way to "cast" this to an element, but the syntax is a bit more involved.

XmlNode node = ((IHasXmlNode) i.Current).GetNode();

Or in most cases you can cast this to XmlElement by

XmlElement node = (XmlElement) ((IHasXmlNode) i.Current).GetNode();

Get a Permission Error When Creating A New Scheduled Task

Symptom:

You tried to create a new Scheduled Task on Windows XP (may be other version of Windows) and you get a Permission Denied error even though you are logged in as the local administrator.

Fix:

The Schedules are stored under C:\Windows\Tasks Unfortunately if you try to alter the permission on the specific folder you may not be able to.

In order to get around this, you can 'Mount a Network Drive" of the path \\[YourComptuerName]\c$\Windows\Tasks

Be sure that Tasks is the top level of the share.

Now you can right mouse click the mounted drive and you can change the permissions.

The Difference Between XmlNode and XmlElement

Symptom:

You are confused why there are XmlNode and XmlElement classes.

Answer:

I was confused by this too. The confusion stems from the fact that if we deal with XML text (especially coming from the understanding of HTML and editing them with Notepad or whatever), what we normally deal with in XML are XmlElements. But in the Xml Standard, even the attributes within an element is a Node.

The detail is that XmlElements are the ones that has tags.

The second source of confusion is that just about everything in Xml is a subclass of XmlNode. That includes attributes in an XmlElement.

We should also note that in System.Xml namespace, XmlElement is a subclass of XmlLinkedNode class which is worthwhile examining in the Microsoft document.

All of this stems from the fact that Xml DOM (Document Object Model) dictates that everything is derived from the single Node class and System.Xml class classes closely follow this hierarchy.

Creating XMLDocument from Program Embedded String in C#

Problem:

You want to quickly create a simple XML document without needing to write data to a file.

Solution:

Use StringReader class and feed that into the Load() function.

 Example Code:
XmlDocument doc = new XmlDocument();
doc.Load(new StringReader("<myNode>Hello</myNode>"));

Update: Well, that was doing too much, Load(string) would have worked just as well.

How To Use InsertAfter() in XmlDocument and Copy Nodes from another XmlDocument

Symptom:

You tried to insert an element (actually it has to be an XmlNode) using InsertAfter() or InsertBefore() method as doing so results in an exception with "The reference node is not a child of this node."

You also want to do the same but take a node from one XmlDocument to another.

Personally I feel that the Xml classes in the .NET Framework is not very intuitive. If you approach them as if they are regular List or Tree class type collection, then you will get a bunch of exceptions.

In terms of InsertAfter() type operations, this is a Node based operation and you normally cannot run InsertAfter() against the XmlDocument object. You must at least go down to the first child of the document, then invoke InsertAfter() at the node with the new node data (to be inserted) and the reference node location. This is most puzzling to me but that's what you need to do.

In terms of copying the data from a node in one document to another, all the nodes created or inserted in one XmlDocument have the membership to that XmlDocument. This is why new nodes must be created from the originating document using CreateElement() type calls. If you are familiar with creating a new row from the DataTable class, then you also know that you have to add that row to the table later is similar to this approach.

To confuse the matter, there are Clone type functions in XmlNode (see note below*). In this context they seem to be of no use (again that's not intuitive to me, since I tend to think if you clone it, I would say you created a detached instance.)

In actuality though, it is an error condition if you simply copy the node to another XmlDocument. This is also separate from where the node resides in overall tree of the nodes. ImportNode() simply buys the membership in another XmlDocument without reserving exactly where your own seat is.

Another important point to remember is that the Node (or Element) you insert to another XmlDocument should be the one returned from the ImportNode() method and not the one you passed to ImportNode(). If you try to insert the node that you passed to the ImportNode() method, you will get an exception "The node to be inserted is from a different document context."

XmlNode useThisNode = doc.ImortNode( myOriginalNode );

Solution Example:

The following code example demonstrates how InsertAfter() should (or at least could) be used, then add yet another node from another XmlDocument (created in-line in the code).

 XmlDocument Insertion Example
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Xml;
using System.Xml.XPath;
namespace XMLDocumentExample
{
class Program
{
static void Main(string[] args)
{
// === EXAMPLE 1: How To Use InsertAfter in XMLDocuments ========================
XmlDocument doc1 = new XmlDocument();
// This XPathNavigator creates a convenient way to Write out the tree on the console.
// we don't use it for nothing else in this example.
XPathNavigator nav1 = doc1.CreateNavigator();
// This creates the root document, which all XML documents will need.
// i.e. <doc1></doc1>
XmlElement e = doc1.CreateElement("doc1");
doc1.AppendChild(e);
// Everything in thid document are the child of the root. I will purposely make a
// Mistake here so that I don't have doc1.2.0 element. I will demonstrate inserting
// doc1.2.0 using InsertAfter.
/*
*
* <doc1>
<doc1.0.0 />
<doc1.1.0 />
<doc1.3.0 />
</doc1>
*
*/
XmlElement c = doc1.CreateElement("doc1.0.0");
e.AppendChild(c);
c = doc1.CreateElement("doc1.1.0");
e.AppendChild(c);
c = doc1.CreateElement("doc1.3.0");
e.AppendChild(c);
Console.WriteLine(nav1.OuterXml); // Will show you you have doc1.0.0, doc1.1.0 and doc1.3.0 now.
// Now we will try to add the missing doc1.2.0 and just as a bonus I will attach doc1.2.1 as the
// child of doc1.2.0
// First let's get to the node AFTER where we will insert doc1.2.0, so that will be doc1.1.0
// Note you can either use /doc1/doc1.1.0 or //doc1.1.0 as the XPath query.
XmlNode nod1_1_0 = doc1.SelectSingleNode("//doc1.1.0");
// This block of code assembles doc1.2.0 node and its child doc1.2.1
XmlElement n2 = doc1.CreateElement("doc1.2.0");
XmlElement n2c = doc1.CreateElement("doc1.2.1");
n2.AppendChild(n2c);
// This is the most tricky important part. You have to ask the parent node to InsertAfter, and not
// directly with doc1.
//XmlNode myParent = nod1_1_0.ParentNode;
//myParent.InsertAfter(n2, nod1_1_0);
XmlNode mp = doc1.FirstChild;
mp.InsertAfter(n2, nod1_1_0);
// The result would look like this:
/*
<doc1>
<doc1.0.0 />
<doc1.1.0 />
<doc1.2.0>
 <doc1.2.1 />
</doc1.2.0>
<doc1.3.0 />
</doc1>
* */
// Write the tree out on the console.
Console.WriteLine(nav1.OuterXml);
// === EXAMPLE 2: Copy A Node from One XMLDocument to Another ==============
// Create the Second XmlDocument, and then copy a node in the second
// document into the first document.
XmlDocument doc2 = new XmlDocument();
doc2.AppendChild(doc2.CreateElement("doc2"));
doc2.FirstChild.AppendChild(doc2.CreateElement("doc2.0.0"));
XPathNavigator nav2 = doc2.CreateNavigator();
Console.WriteLine(nav2.OuterXml);
XmlNode node_2_0_0 = doc2.SelectSingleNode("//doc2.0.0");
// Note that this does not append the node, it simply says that
// a foreign node can now belong to the first document.
XmlNode importableNode = doc1.ImportNode(node_2_0_0, true);
doc1.SelectSingleNode("/doc1/").AppendChild(importableNode);
Console.WriteLine(nav1.OuterXml);
Console.ReadLine();
}
}
}

*Note: Cloning of a node is required when you need to make a copy of a node and insert that back into the same XmlDocument. It appears that you cannot make a reference to a node and insert that (of course, if you can do that then there is a side effect of both node data changing if you edit the content of one node.)

Handling Customer or Press Complaints of Issues

Symptom

Customers are equivocally complaining about some aspect of your product.

Solution:

I just watched the 15-minute video of Steve Jobs iPhone 4 July 16th Press Conference and I was very impressed the way he handled the situation. So next time something like that happens to me I am going to note what I've learned from this lesson.

Before you contact the customer the following preparation should be made:

• Gather as much facts and historical information about the performance of the product, for example issue call or email history from the customer or other customers, and the same for other similar customer set. Note: This is why you should rigorously record all issues using some type of database.
• Know the workarounds. In other words, don't even contact the customer or give a conference before you know the workarounds.
• Be prepared with the software patch but do not release it before you contact the customer. Since visibility of your effort is important, do not install it until the customer can expect to see the improvements.

While contacting the customer:

• First, create a less stressful and friendly atmosphere. Tell jokes, other stuff.
• Begin with a positive tone by saying that your product is the best available, worked hard on it by best people and if you do have good reviews and so forth, tell them about that.
• Next, indicate that you are puzzled why the customer is making such claims to the best product available.
• But show that the customer is always right, so you say that "we have started to look into it."
• First, say: "Nothing is Perfect. You know that, and I know that."
• With not-so-surprised expression, talk about similar situations on other similar products. Tell customers that it is not just "our" product that is having the issue. Bring up the facts by video or verifiable statistics.
• With a surprised expression (if your number is actually lower), explain about the claim by other verifiable (or at least authentic) figures such as previous dissatisfaction statistics (number of issue tickets, number of returns).
• If you have a patch, inform the customer about it and then release or install it.
• Deny most of allegations, but admit a little
• Offer workarounds until the issue is completely solved, including refunds if that's possible.

Unable to switch the encoding at System.Data.SqlClient.SqlConnection

Symptom:

You have done the following:

• You have set up an SQL table with XML data type in it.
• You have a Strongly Typed DataSet
• You try to write XML from a C# String
• You get the following error

You have a DataSet unable to switch the encoding at System.Data.SqlClient.SqlConnection

What Fixed It For Me

The part of the original XML which went into the string had

in the XML heading

by simply switching this to

That fixed the problem.

Note:

• In my case the real issue was that the actual strung was encoded in Unicode (UTF-16) but the XML header said utf-8. That mismatched.
• In your case the actual string may be encoded in some other format such as utf-8. Be sure to check. If the real encoding is the issue then you could re-encode your string from what you have into UTF-16. Look for System.Text.Encoding class on MSDN for further information.

Finding Uptime Using 'net statistics server' on Windows 2008 Server Does Not Work

Symptom:

You type in to a CMD.exe the following command,

net statistics server

and it was showing the uptime of the server on your Windows 2003 server, but when you tried this on a 64-bit Windows 2008 Enterprise R2, it does not show correct time.

Fix:

Though bizarre, try

net statistics workstation

See this article: http://support.microsoft.com/kb/2077595

Please note that in my situation the server is 2008 R2 SP2 but it still does not work with 'net statistics server' Since the workstation command works, I am OK by it.

Ignoring .MySCMServerInfo in Visual Studio Web Project

Problem:

• You are using Microsoft Visual Studio 2005, 2008, or 2010 with Seapine SurroundSCM.
• When you are working with Web project you see annoying .MySCMServerInfo file in each directory and furthermore, VisualStudio make you check them in when SurroundSCM does not want you to check them in. You don't get this issue when you are working on non-Web (e.g., "desktop" or class library) projects.

Solution:

Please see this StackOverflow Article: http://stackoverflow.com/questions/187359/hidding-source-control-files-within-visual-studios-solution-tree

Just in case the answer link is lost in the future, the idea is to add the Ignorable File Key (yes, Key with the name of the file(s) you want to ignore and not the Data) with file names you want to ignore from the source control. It works on Version 8.0 (VS 2005), Version 9.0 (VS 2008) and Version 10.0 (VS2010).

[HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\VisualStudio\10.0\Packages\

{8FF02D1A-C177-4ac8-A62F-88FC6EA65F57}\IgnorableFiles\.MySCMServerInfo]

ASP.NET MVC2 New Project Fails To Build with 'The type or namespace name 'Controllers' does not exist in the namespace..."

Symptom:

You have just created a brand new MVC2 Web project and tried to compile and run without anything modified and you get the following error:

The type or namespace name 'Controllers' does not exist in the namespace

What Fixed For Me:

The Unit Test project does not have the reference to the Main project. Just go to the Reference folder of the Unit Test Project and make a reference to the Main project.

Windows Server Hosts File Ignored or Not Looked Up

Our Symptom:

We often need to circumvent customer/client's DNS or supplement them since FQDN is a very stringent requirement for Microsoft SQL Server Mirroring technology.

One of the tricks we use is to edit the local hosts file C:\Windows\System32\Drivers\Etc\Hosts to yield correct FQDN internally.

At one point I have noticed that no matter what I enter the information there, they are ignored.

Also you may have noticed this when the SQL Server Management Studio takes a long time to come up especially in a closed network where there is actually no "outside" route. As I have posted previously, this is due the fact that SQL Server Management Studio performs Certificate Revocation List (CRL) lookup. We usually add

127.0.0.1 crl.microsoft.com

in our local Hosts file so that CRL lookup will purposely fail in order to get the SSMS to come up quicker.

What Worked for Me:

Quick Test: Try to see in your service controller if "DNS Client Service" that is running. If you turn it off, the system starts to look up names in the local Hosts file.

Next you should look your Windows registry.

HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\Tcpip\Parameters

In there you will find the value

UseDomainNameDevolution

If this is set to 1, your Hosts wont' work. Just reset this to zero (0)

For additional information on this, find here: http://technet.microsoft.com/en-us/library/cc766230(WS.10).aspx

Google App iPhone Calendadar Does Now Show All Shared Calendars

Symptom:

You are syncing to the Calendar on iPhone, and you want to see all of the shared calendars. You have accepted the invitations to new shared calendars and you do and can see them while you are using a desktop web browser.

Solution:

Go to the following page, with (yourdomain) replaced with your actual domain, for example, "example.com"

https://www.google.com/calendar/hosted/(yourdomain)/iphoneselect

You get [Microsoft][ODBC Driver Manager] Data source name not found and no default driver specified on 64-bit Windows

Symptom:

When you try to connect to an SQL Server on Windows 2008 64-bit server running 32-bit application you get the following error

[Microsoft][ODBC Driver Manager] Data source name not found and no default driver specified

And you cannot also find the DSN in the list of ODBC DSNs

Cause:

You must configure the 32-bit ODBC driver. You have been configuring the 64-bit ODBC driver all along. This is not very clear to you (or me), and there is no explicit control panel items for both.

To start a 32-bit ODBC driver,

C:\Windows\SysWOW64\odbcad32.exe

HP ILO 100 KVM Invalid username/password Error

Symptom

You have an HP DL Series server such as DL150. You did purchase and installed the KVM license key, it did work for some time but now whenever you connect you get

"Invalid username/password. You have been disconnected"

Cause


Something has gone wild in the ILO Flash memory.


Fix

ssh as an admin into the ILO interface. For most cases you should use putty program which is free and downloadable.

Then type in the following commands:

show map1 license

(just to play it safe and copy and paste this into a suitable text file on your computer)

reset map1 license

Session will then hang.

Log back into the web utility and the KVM feature should now work.

Originally Appeared on: http://forums13.itrc.hp.com/service/forums/questionanswer.do?admit=109447627+1275318554605+28353475&threadId=1404325

IMAP Problem with iPhone or iPOD for Google App

Symptom:

You are hosting your domain with Google App and allowing people to get emails via IMAP.

You get a complaint from a user stating that:

• They can read their email on the web
• They cannot suddenly access their email on iPhones even though IMAP and POP are enabled in their profile.
• May get "Cannot Connect Using SSL" message.

Possible Root Cause:

• Google has disabled IMAP (and POP) access probably due to some security issues such as repeated IMAP access with incorrect password.
• If this is the cause you won't be able to access your Google App email via IMAP or POP either, but most iPhone users don't realize this is happening since they may never use IMAP or POP clients on their home computers.

Fix:

• Try doing "reset Captcha" and this should re-enable IMAP or POP thereby allow the access from iPhone or iPod Touch

To do this, use the following url by replacing with your own domain.

https://www.google.com/a/yourdomain.com/UnlockCaptcha