Joshua Watkins, Piotr Woloszyn
Just before Christmas Josh Watkins and me had to face the interesting task of removing the dependency on Microsoft Web Services Enhancements (WSE) 3.0 in one of the projects that we are working on currently. The task consisted of two parts: signing the soap requests and validating incoming soap responses. Thanks to Robbie’s spike for signing requests we managed to quickly finish the first task. I might blog about this as well soon. The second one is a different story. As this turned out to be a pain in the butt and we couldn’t find easily comprehensive information about verifying XML’s digital signatures on the web we decided to share our finding with wider audience and hopefully help someone who has to face the same challenge. Alors…
Josh: I would just like to add that it was a really really … really big pain in the butt.
XML digital signatures (XMLDSIG) allow you to verify that data was not altered after it was signed. Working in the Microsoft .NET 2.0 world it wouldn’t be difficult as we could use the SignedXml class as described here:
// Verify the signature of an XML file against an asymmetric
// algorithm and return the result.
public static Boolean VerifyXml(XmlDocument Doc, RSA Key)
{
// Create a new SignedXml object and pass it
// the XML document class.
SignedXml signedXml = new SignedXml(Doc); // Find the "Signature" node and create a new
// XmlNodeList object.
XmlNodeList nodeList = Doc.GetElementsByTagName("Signature");
// Load the first <signature> node.
signedXml.LoadXml((XmlElement)nodeList[0]);
// Check the signature and return the result.
return signedXml.CheckSignature(Key);
}
Unfortunately the signed soap responses were hashed with SHA256 algorithm defined as:
<signaturemethod algorithm="http://www.w3.org/2001/04/xmldsig-more#rsa-sha256">
And that algorithm is unsupported:
With the 2.0 release of the CLR, there is no way to use the RSA-SHA256 signature type with SignedXml. Adding a SHA-256 CryptoServiceProivder implemenation is high on our list of items to look at in the next version, which should enable this scenario.
- Shawn Farkas, Microsoft, 09/09/2007
Josh: When we first tried using the SignedXML method we would get a malformed Reference Element error (or something like that). Which as you can imagine, isn’t really all that helpful or descriptive.
The only available option was to verify the signature manually. Before we will get to details how to do it let’s look at the XML of the SOAP response coming back to us. As we can see there is Signature element which has the children SignedInfo and SignatureValue. SignedInfo holds information about which parts of the message have been signed, with which algorithm, and the hash of the signed element. No one will be able to tamper with the content of the nodes (in this example we sign soap:Body, wsu:Timestamp, wsse:BinarySecurityToken) without being detected through varying hash values. The SignatureValue is the content of the SignedInfo node hashed and signed with the private key of sender, so we can trust the content of that node as well. The whole problem is in the step which has to be performed before hashing: Canonicalization of the XML. Getting the process right for c14n was the most time consuming part.
Now more details and some code. Let’s assume that we implement System.Web.Services.Protocols.SoapExtension and will verify the message in overriden ProcessMessage(SoapMessage message) method, stage SoapMessageStage.BeforeDeserialize. Let’s assume as well that we have the content of the soap message as string (messageContent). To verify the digital signature of an XML document:
1. Create XmlDocument using the messageContent:
XmlDocument xDoc = new XmlDocument();
xDoc.PreserveWhitespace = false;
xDoc.LoadXml(messageContent);
2. Load the server certificate from file system:
X509Certificate2 serverCertificate = new X509Certificate2(pathToServerCert);
3. Now let’s validate message signature. Move to SignatureValue node using the lovely XPathNavigator:
XPathNavigator nav = xDoc.CreateNavigator();
nav.MoveToFollowing("SignatureValue", "http://www.w3.org/2000/09/xmldsig#");
4. The signature value requires that all of the \n inserted into the value are removed before we processed it:
string signatureValue = Regex.Replace(nav.InnerXml.Trim(), @"\s", "");
5. The signature value is saved in the messages encoded in base64. Therefore to get the actual signature value we have to convert back from base64:
byte[] sigVal = Convert.FromBase64String(signatureValue);
6. The SignedInfo XML block is extracted from the greater document. Then we take out all of the namespaces used in all of the parent nodes of SignedInfo and then reinsert them into the SignedInfo block. Without this your validation will FAIL.
Josh: And our validation did indeed fail every time without this. J
XmlNode signedInfo = xDoc.GetElementsByTagName("ds:SignedInfo")[0];
Hashtable ns = RetrieveNameSpaces((XmlElement)signedInfo);
InsertNamespacesIntoElement(ns, (XmlElement)signedInfo);
where:
public static Hashtable RetrieveNameSpaces(XmlElement xEle)
{
Hashtable foundNamespaces = new Hashtable();
XmlNode currentNode = xEle;
string name = null;
while (currentNode !=null)
{
//add namespace for current nodes namespace if it has one.
if (currentNode.NodeType == XmlNodeType.Element && !String.IsNullOrEmpty(currentNode.Prefix))
{
if (!foundNamespaces.ContainsKey("xmlns:" + currentNode.Prefix))
{
foundNamespaces.Add("xmlns:" + currentNode.Prefix, currentNode.NamespaceURI);
}
}
//now we add namespaces for any attributes that this node may have.
if (currentNode.Attributes !=null && currentNode.Attributes.Count>0)
{
for (int i=0; i< currentNode.Attributes.Count;i++)
{
if (currentNode.Attributes[i].Prefix.Equals("xmlns") || currentNode.Attributes[i].Name.Equals("xmlns"))
{
if (!foundNamespaces.ContainsKey(currentNode.Attributes[i].Name))
{
foundNamespaces.Add(currentNode.Attributes[i].Name, currentNode.Attributes[i].Value);
}
}
}
}
currentNode = currentNode.ParentNode;
}
return foundNamespaces;
}
public static void InsertNamespacesIntoElement(Hashtable namespacesHash, XmlElement node)
{
XPathNavigator nav = node.CreateNavigator();
if (String.IsNullOrEmpty(nav.Prefix) && String.IsNullOrEmpty(nav.GetAttribute("xmlns","")))
{
nav.CreateAttribute("", "xmlns","",nav.NamespaceURI);
}
foreach (DictionaryEntry namespacePair in namespacesHash)
{
string[] attrName = ((string)namespacePair.Key).Split(':');
if (attrName.Length > 1 && !node.HasAttribute(attrName[0]+":"+attrName[1]))
{
nav.CreateAttribute(attrName[0], attrName[1], "", (string)namespacePair.Value);
}
}
}
7. Canonicalize the signedInfo:
Stream signedInfoStream = canonicalizeNode(signedInfo);
where:
public static Stream canonicalizeNode(XmlNode node)
{
XmlNodeReader reader = new XmlNodeReader(node);
Stream stream = new MemoryStream();
XmlWriter writer = new XmlTextWriter(stream, Encoding.UTF8);
writer.WriteNode(reader, false);
writer.Flush();
stream.Position = 0;
//To ensure that the XML is properly formatted we use this transform on it
// before creating the hash of the SignedInfo block.
XmlDsigC14NTransform transform = new XmlDsigC14NTransform();
transform.LoadInput(stream);
return (Stream)transform.GetOutput();
}
8. The next step is to compute the hash of the canonicalized signedInfo:
SHA1 sha1 = SHA1.Create();
byte[] hashedSignedInfo = sha1.ComputeHash(signedInfoStream);
9. The last step of the validation of the message signature is to verify the hash using a crypto service provider via the sender’s public key:
string oid = CryptoConfig.MapNameToOID("SHA1");
RSACryptoServiceProvider crypto = (RSACryptoServiceProvider)cert.PublicKey.Key;
bool isMessageSignatureValid = crypto.VerifyHash(hashedSignedInfo, oid, sigVal);
10. OK, if we got so far it means that nobody tempered with the SignedInfo node. Now we have to validate if no one tempered with content of signed elements by inspecting references and hashes in the SignedInfo. That’s the place where the SHA256 algorithm is used. First let’s load the message references from the SingedInfo node:
XmlNamespaceManager man = new XmlNamespaceManager(xDoc.NameTable);
man.AddNamespace("soap", "http://schemas.xmlsoap.org/soap/envelope/");
man.AddNamespace("wsse", "http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd");
man.AddNamespace("ds", "http://www.w3.org/2000/09/xmldsig#");
// A bit of xpath to only get the reference nodes in the security header.
XmlNodeList messageReferences = xDoc.SelectNodes("//soap:Header/wsse:Security/ds:Signature/ds:SignedInfo/ds:Reference", man);
11. For each of the references we shall perform validation (steps 12-17). First we will look up the reference listed in SignedInfo and retrieve the correspodning node from the xDoc.
XPathNavigator elementNav = xEle.CreateNavigator();
string elementID = elementNav.GetAttribute("URI", "");
//We need to remove the hash. :)
if (elementID.StartsWith("#"))
{
elementID = elementID.Substring(1);
}
12. Now we find the node associated with the id.
XmlElement referencedNode = retrieveElementByAttribute(xDoc, "wsu" + ":Id", elementID);
where:
public static XmlElement retrieveElementByAttribute(XmlNode xDoc, string attributeName, string attributeValue)
{
ParameterValidation.Validate("xDoc", xDoc);
ParameterValidation.Validate("attributeName", attributeName);
ParameterValidation.Validate("attributeValue", attributeValue);
XmlElement foundElement = null;
foreach (XmlNode node in xDoc)
{
if (node.HasChildNodes)
{
foundElement = retrieveElementByAttribute(node, attributeName, attributeValue);
}
if (foundElement == null && node.Attributes != null && node.Attributes[attributeName] != null && node.Attributes[attributeName].Value.Equals(attributeValue))
{
foundElement = (XmlElement)node;
break;
}
if (foundElement != null)
{
break;
}
}
return foundElement;
}
13. Then we will incorporate the namespaces from the parent node of the retrieved node
InsertNamespacesIntoElement(RetrieveNameSpaces((XmlElement)referencedNode.ParentNode), referencedNode);
14. Now we will canonicalize the node with the specified canonicalization method.
Stream canonicalizedNodeStream = canonicalizeNode(referencedNode);
15. Create the proper hash algorithm object and compute the hash of the signed node:
elementNav.MoveToFollowing("DigestMethod", "http://www.w3.org/2000/09/xmldsig#");
HashAlgorithm hashAlg = (HashAlgorithm)CryptoConfig.CreateFromName(elementNav.GetAttribute("Algorithm", ""));
byte[] hashedNode = hashAlg.ComputeHash(canonicalizedNodeStream);
16. Load the digest value and decode its value using base64 encoding:
elementNav.MoveToFollowing("DigestValue", "http://www.w3.org/2000/09/xmldsig#");
byte [] digestValue = Convert.FromBase64String(elementNav.InnerXml)
17. If the hashedNode array and the digestValue array are equal then the verification of the reference from the SignedInfo is positive! After repeating this for all the references the verification of the digital signatures of XML documents, without using WSE3 is done.
Josh: It all seems so easy now. I think the hardest part of this entire process is never really knowing if you are on the right path until you get it completely right. Hopefully this tutorial will help someone else avoid the pain and suffering that we had to endure.
Some other useful links:
Apache XML Security
An Introduction to XML Digital Signatures
Bouncy Castle
PS. REST rulez!
Popularity: 62% [?]
Share This
I came across a strange behaviour of Selenium and wanted to share my pain with someone really. But first couple of words of introduction. I use Selenium in my current webapp project which I am working on. The application uses quite a lot of javascript (Dojo framework mostly) which can be tested using D.O.H. framework but the Selenium is giving us the final sanity checks, acts as integration test framework and may be used by product owner in an agile environment to drive acceptance on user stories via automated acceptance tests.
It is an example how to use the 
If you have to use Outlook I believe you deserve better search features than the features available for you by default. Many of you know the 
So… There is a new project 
I believe in testing, I really do. Combined with 
I can recall a conversation with 
Some of you who use Eclipse 3.3 with Sun JVM might notice that it crashes from time to time. Those who have many plugins certainly experienced this more often. Problem and solution for this 