Archive for the ‘CodeProject’ Category

Who knows of the .Net Secure Strings?

Thursday, January 21st, 2010

[Warning this is not new stuff - but shouldn't be overlooked if you need to secure sensitive data in your application]

Isn’t “Secure String” an oxymoron for .Net? So if we are thinking about securing some sensitive data in say C or C++ its relatively simple load it into a char array memory and encrypt it, wiping the memory out after the information has been loaded.

Now try that with .Net! From the Microsoft site:

A String is called immutable because its value cannot be modified once it has been created.

So how can you destroy one? Set it to empty? Well simply put you can’t :-) . Once your string is not longer referenced, or worse yet your object containing the string its time for the Garbage Collector to come and do its work. The problem is if your object has been around long enough to get into Generation 1 or 2 then it is going to take a bit longer.

Hmmm so in translation if you keep a password, Credit Card, encryption key or some other sensitive text in memory as a string you cant destroy it (think memset for us oldies!). Only the GC can free the memory for you, and you are dependent on HOW it frees that memory. I personally don’t know for a fact if it memsets it to blank, or just dereferences the pointer. However I would be willing to bet it is the option that requires the least amount of work and that doesn’t bode well for controlling the exposure of our sensitive data.

Plainly that proverbially sucks!

Enter the “SecureString” class, from the MS site it says:

“Represents text that should be kept confidential. The text is encrypted for privacy when being used, and deleted from computer memory when no longer needed.”

Wow doesn’t that just sound like the ticket we need! Secure, Encryption, delete from memory – how fantastic! Uh oh keep reading the remarks:

“Your application can render the instance immutable and prevent further modification by invoking the MakeReadOnly method.

Use appropriate members of the System.Runtime.InteropServices..::.Marshal class, such as the SecureStringToBSTR method, to manipulate the value of a SecureString object.”

BSTR – oh I feel the COM headache coming back!

Actually its really not that bad, but its definitely not a straight swap for a System.String. See some example code below:


using System;
using System.Text;
using System.Runtime.InteropServices;
using System.Security.Cryptography;
using System.Security;

namespace CSharpHacker.Utilities
{
   /// <summary>
   /// Demo helper class for <see cref="SecureString"/>
   /// </summary>
   public class SensitiveDataHelper
   {
      private SecureString sensitiveData;

      /// <summary>
      /// Gets or sets the sensitive data class from helper
      /// </summary>
      /// <value>The sensitive data container.</value>
      public SecureString SensitiveData
      {
         get { return sensitiveData; }
         set { sensitiveData = value; }
      }

      /// <summary>
      /// UNSECURE: Converts the secured sensitive data into an unsecured string.
      /// </summary>
      /// <remarks>
      /// This is a dangerous function that converts secured information into
      /// unsecured data.
      /// </remarks>
      /// <returns>String representing the secured data</returns>
      public string SensitiveDataToString()
      {
         IntPtr ptr = Marshal.SecureStringToGlobalAllocUnicode(this.sensitiveData);
         try
         {
             // Unsecure managed string
             return Marshal.PtrToStringUni(ptr);
         }
         finally
         {
             Marshal.ZeroFreeGlobalAllocUnicode(ptr);
         }
      }

      /// <summary>
      /// UNSECURE: Base64s encodes a SHA512 has of the sensitive data
      /// </summary>
      /// <returns>SHA512 hash value</returns>
      public string Base64SensitiveDataHash()
      {
         IntPtr bstr = Marshal.SecureStringToBSTR(sensitiveData);
         try
         {
            // Pretend simple hash function that returns a string - this is fake just for readability!!
            string output = Marshal.PtrToStringBSTR(bstr);

            Marshal.FreeBSTR(bstr);

            SHA512 sha = new SHA512Managed();
            byte[] result = sha.ComputeHash(Encoding.UTF8.GetBytes(output));
            return Convert.ToBase64String(result);
         }
         finally
         {
            Marshal.ZeroFreeBSTR(bstr);
         }
      }

      /// <summary>
      /// Loads the sensitive data into a <see cref="SecureString"/>
      /// </summary>
      /// <param name="sensitiveInformation">The sensitive information to protect.</param>
      public void LoadSensitiveData(char[] sensitiveInformation)
      {
         try
         {
            using (SecureString securePassword = new SecureString())
            {
               foreach (char c in sensitiveInformation)
               {
                  securePassword.AppendChar(c);
               }
               securePassword.MakeReadOnly();
               this.sensitiveData = securePassword.Copy();
            }
         }
         finally
         {
            // discard the char array
            Array.Clear(sensitiveInformation, 0, sensitiveInformation.Length);
         }
      }

      /// <summary>
      /// Loads the sensitive data into a <see cref="SecureString"/>
      /// </summary>
      /// <param name="sensitiveInformation">The sensitive information to protect.</param>
      public void LoadSensitiveData(string sensitiveInformation)
      {
         char[] sensitive = new char[sensitiveInformation.Length];
         sensitiveInformation.CopyTo(0, sensitive, 0, sensitive.Length);

         LoadSensitiveData(sensitive);
      }

      /// <summary>
      /// Loads the sensitive data into a <see cref="SecureString"/>
      /// </summary>
      /// <param name="sensitiveInformation">The sensitive information to protect.</param>
      public void LoadSensitiveData(StringBuilder sensitiveInformation)
      {
         char[] sensitive = new char[sensitiveInformation.Length];
         sensitiveInformation.CopyTo(0, sensitive, 0, sensitive.Length);

         LoadSensitiveData(sensitive);
      }

      /// <summary>
      /// Creates the secure string from string.
      /// </summary>
      /// <param name="unprotectedSensitiveInformation">The unprotected sensitive information.</param>
      /// <returns></returns>
      public static SecureString CreateSecureStringFromString(string unprotectedSensitiveInformation)
      {
         char[] unprotectedSensitive = unprotectedSensitiveInformation.ToCharArray();
         SecureString secureInformation = new SecureString();
         try
         {
            foreach (char c in unprotectedSensitive)
            {
               secureInformation.AppendChar(c);
            }
            secureInformation.MakeReadOnly();
            return secureInformation;
         }
         finally
         {
            // discard the char array
            Array.Clear(unprotectedSensitive, 0, unprotectedSensitive.Length);
         }
      }
   }
}

A word of caution to the above code:

  • SensitiveDataToString – is considered insecure as it returns a string of the encrypted data. The same data we are trying to encrypt! However it is a commonly requested function, and so there is the implementation.
  • Base64SensitiveDataHash – is considered insecure as it currently uses a temporary string (:-( ), at this time I don’t have a way to converts a BSTR into a an array without going through a string first. One way would be to process it prior to being made ReadOnly, or alternatively someone can write a comment how to convert a C# BSTR into a byte array!

So even with all those disclaimers running a program that takes input will still a ‘leak information’ for the System.String. Specifically the tricky area is how do you get them into you program in the first place? Read them from a database, WinForm user input, or a web page? Kinda tricky :-) ! If you search the web there are implementations of  secure login controls that build it up character by character, but certainly something to think about.

So how Secure is “SecureString”? The answer is “it depends”, but reasonably secure and a heck of a lot better than System.String. A while ago there was a big storm about tools that can connect to the process and decrypt your SecureStrings. The best rebuttal to this I’ve seen can be read about in [SecureString Redux]. I definitely recommend reading this.

Now I have to say I’ve been meaning to write this for some time now! Hopefully this helped raise awareness of the string leakage risks in the .Net language and ways to help minimize the string information leak scenario.

Potential enhancements to the helper class would be:

  • Make the Hashing really secure, and allow a “HashAlgorithm” to be passed in
  • Allow external encryption
  • Allow secure serialization of data (via the encryption)

Gareth

How security is very much like MMA

Sunday, September 20th, 2009

It occurred to me after following the most recent UFC MMA (via the web blogs rather than PPV as I’m still too cheap!) that security and MMA have a lot in common. More precisely the fighters in a stable as very similar to security algorithms or process.

Once a fighters weakness has been exposed there is really nothing you can do to unhide that weakness. You could have the best fighter in the world one day, then the weakness is exposed… You are in trouble!

Security is very much the same. You can perform all the scans, probes, fuzzes, code reviews and feel confident (well as confident anyone does in the security world!) that you are pretty well covered. One revelation a day later can completely invalidate your expectations, and you have to completely start over. Sometimes it is a slow build up, other times it is the equivalent of a bomb.

Bottom line is once a weakness has been exposed you need to:

  • See if it can be simply covered
    • Fighter can learn to defend against take downs (or not get hit in the head :-) )
    • Algorithm can be enhanced to extend its life DES==>3DES
  • Relegate
    • Fighter acts as the ‘gatekeeper’ to the higher competition levels
    • Algorithms security clearance has been lowered, it cant be used in the more secure areas. Examples of this are theoretical discoveries that are likely to result in the actual weakness discover some time later.
  • Retire
    • Fighter retires, becomes a commentator!
    • Algorithm depreciated as it is shown to be fundamentally insecure, now studied in university to show the weakness that designers need to be aware of. Think WEP!

If the weakness is known it is natural the opponent will attempt to get a competitive advantage using it. The longer the weakness is known the more adept the opposition will be at exploiting it.  This is true for both MMA & security!

Companies running a SDL are the equivalent to the fighters stable. It is their job to recognize the weaknesses and manage the processes and algorithms so any weaknesses are covered or retired before they become a major problem.

Gareth

SQLite for C# – Part 8 – Loading CSV/Pipe into SQLite via command line

Saturday, September 19th, 2009

Ever wondered how hard it would be to load a CSV file into a SQLite database. I know how I would do it in code, no rocket science needed there! However in this case I wanted to really know the speed of doing this natively and really didn’t want to code anything!

So looking at what SQLite3.exe has too offer it pretty much supports it out of the box. Very nice :-)

Requirements:

  • Loading speed
  • Making the data to consuming applications available asap

While I love C# and frankly its hard to go back to C or C++, sometimes performance trumps the creature comforts we have become accustomed to.

Note: I did this without circling back to a C# implementation as I know the data and performance requirements  are tight and in this case I wanted max performance with no code! The biggest factor to a successful implementation is to ensure you use the tools best for the job, not just the ones you favor in that specific year.

So first things first – create a table to take the input

DROP TABLE IF EXISTS BookSales;
CREATE TABLE IF NOT EXISTS BookSales
(
   Store    int
  ,Date     varchar
  ,OrderReference varchar
  ,Line     int
  ,BookISBN varchar(14)
  ,Quantity int
  ,Price    int
, Primary Key (OrderReference,Line)
);

Next is the magic. We need to load the CSV into the table:

.separator "|"
.import BookSales.txt BookSales

Wow that was easy :-) . You can see we set the separator to be a pipe rather than comma in this case, then the import.

.IMPORT [FileName] [Table]

Now the database is ready to be queried! But if we want to take it just one stage further:

.output SummaryBookSales.csv
SELECT Store, Date, BookISBN, SUM(Quantity), SUM(Price)
FROM BookSales
GROUP BY Store, Date, BookISBN;

Now we output the results of our simple aggregation into a pipe separated output file.

Tying this all together in a single configuration file, which we will call “BookAnalysisLoader.sql”, gives us:

DROP TABLE IF EXISTS BookSales;
CREATE TABLE IF NOT EXISTS BookSales
(
   Store    int
  ,Date     varchar
  ,OrderReference varchar
  ,Line     int
  ,BookISBN varchar(14)
  ,Quantity int
  ,Price    int
, Primary Key (OrderReference,Line)
);

.separator "|"
.import BookSales.txt BookSales

.output SummaryBookSales.csv
SELECT Store, Date, BookISBN, SUM(Quantity), SUM(Price)
FROM BookSales
GROUP BY Store, Date, BookISBN;
.exit

The last piece of the puzzle is the final execution:

sqlite3.exe BookSalesAnalysis.db3 < BookAnalysisLoader.sql

Now we have a newly created database with our analysis data in it, and we have a summary CSV file generated from the output. So we can load the CSV into Excel or another DB, or directly interrogate the DB for more analytical information – and all without coding!

Related Links:

Better way to determine and police Password Strengths

Sunday, September 13th, 2009

Perhaps my Google search mo-jo has been acting up, but I could not find a good strong C# implementation for strong passwords (in fact I really couldn’t find much outside of logical cut & paste of implementations of random Information entropy implementations) . They were all predicated on the relatively standard assessment that all submitted passwords are random – uh huh!

For starters I recommend reading the article [http://en.wikipedia.org/wiki/Password_strength]. This is a good article covering the relative strengths of passwords, and gives a guide for determining the strength of a random password and a human derived password.

The major problem with passwords are that humans need to remember them, or they write them down. In an interesting technology twist historically you only used to have to worry about your co-workers having access/abusing your password because there was implicit physical security in place – you could only log on if you were physically in the office.  As such at that time your biggest threat was your co-workers, unfortunately the secondary defense of physical location has effectively been removed with the internet and VPN technology.  So now your threat count has increased from the people you work with to the entire world! Add to this these people are financially motivated and can directly target you – its a whole lot scarier out there now!

So before jumping into the implementations we need to go through well known things to avoid to help improve password strength:

  • Avoid sequences – keyboard or alphabet based (abcd, qwert, 1234, !@#$% etc)
  • Avoid dictionary words, especially common ones! Be aware that common misspellings are also used in dictionary based attacks – so unless your misspelling is VERY unusual then you can expect it to be in a dictionary!
  • Avoid leet/1337 password substitution of words (eg P@ssw0rd, M1cr0$0ft, 0\/\/n3d). Again these are now all in dictionaries, so while it may be harder to brute force – they are pretty trivial for a dictionary attack. Of course it doesnt hurt to be 1337, but it just really doesn’t help defend a targeted attack.
  • Avoid team names, socials, license names etc.

Things to avoid to minimize compromise exposure:

  • Use different passwords for different online accounts
  • Avoid using information about you that can be readily be found on the web as a password reset scheme. DOB, where you were born, school name etc.
  • If any account needs the most rigorous password control it is your email account. Nearly every online system ties back to an email account. If you need to reset a password, it normally goes to your email address. If that is compromised then that is really the opening of Pandora’s box.

Alright, lets start with the weakest ‘safe’ approach – Information entropy:

  • This strength calculation only holds true for ‘random’ passwords. No human (at least that I know) can really generate a random password on their own. The best approach that I’m aware of is to start up notepad and get your two year old to start smacking your keyboard. Then take this text and randomly change case of characters and inserting special characters. Unfortunately this is still weak because we have 2 hands and the keyboard is naturally divided into where your hands go. This generation is not as randomly distributed as people would think – nor would I recommend it! But at least you have a starting point, but then you have to write it down!
  • [0-9] – 10 possible symbols per character – 3.32 bits of base2 log entropy
  • [a-z] – 26 possible symbols per character- 4.7 bits of base2 log entropy
  • [A-Z] – 26 possible symbols per character- 4.7 bits of base2 log entropy
  • [A-Z, 0-9] – 36 possible symbols per character- 5.17 bits of base2 log entropy
  • [A-Z,a-z] – 52 possible symbols per character- 5.7 bits of base2 log entropy
  • [A-Z, a-z, 0-9] – 62 possible symbols per character- 5.95 bits of base2 log entropy
  • [A-Z, a-z, 0-9, Special] – 94 possible symbols per character – 6.55 bits of base2 log entropy

So we can see that having a strong password using completely random information will be hard to generate on our own, yet this approach is what is what is most commonly used to in web applications to determine password strength. This is not strong enough because humans are naturally not random. Using this theory the following non-random passwords generate results that imply the passwords are strong:

  • 12345678901234567890 – 20*3.32 => 66.4 bits of entropy
  • !!!!!!!!!!!!!!!!!!!! – 10 * 6.55 => 65.5 bits of entropy
  • !@#$%^&*() – 10 & 6.55 => 65.5 bits of entropy
  • qwertyuiop[]qwertyuiop[] = 24 * 6.55 => 157.2 bits of entropy.

The more astute among us will see the last two passwords were generated by running your finger across the a keyboard line of on a US keyboard. To enter the 24 characters password took under 3 seconds. So if anyone saw someone entering a password like this at work or in a library – its pretty easy to duplicate. Plainly you can see that with human users they are going to opt for the easiest way to remember and enter a password – this will never be random!

So to help avoid our users from becoming victims we have to try to take away the ‘easy’ passage from them. We have to assume the password is not going to be mathematically random – so we need to start from a different position. We have to ensure we remove the human weaknesses that other ‘black hats’ are looking to exploit.

So going back to the beginning of the article we are going to create an interface to define a ‘password policy’ that provides us a way to help enforce a stronger passwords – or  at least allows systems to setup a common language for handling passwords.

   /// <summary>
   /// Interface for defining a password policy
   /// </summary>
   /// <remarks>
   /// This security policy determines whether passwords
   /// meet pre-determined complexity requirements.
   ///
   /// If this policy is enabled, passwords must meet the
   /// following minimum requirements:
   ///
   /// Not contain the user's account name or parts of the
   /// user's full name that exceed four consecutive
   /// characters.
   /// Be at least <see cref="MinimumPasswordLength"/>
   /// characters in length
   /// Contain characters from three of the following
   /// four categories:
   /// English uppercase characters (A through Z)
   /// English lowercase characters (a through z)
   /// Base 10 digits (0 through 9)
   /// Non-alphabetic characters (for example, !, $, #, %)
   ///
   /// Complexity requirements are enforced when passwords
   /// are changed or created.
   /// </remarks>
   public interface IPasswordPolicy : IPolicy
   {
      /// <summary>
      /// Indicates the minimum password strength index for
      /// this policy (see PasswordStrengthIndex)
      /// </summary>
      /// <remarks>
      /// This value is based of a calculation of
      /// information entropy after sequences
      /// and dictionary words have been
      /// removed.
      /// </remarks>
      /// <value>
      /// The minimum index of the password strength.
      /// </value>
      PasswordStrengthIndex MinimumPasswordStrengthIndex
      {
         get;
         set;
      }

      /// <summary>
      /// Gets or sets the minimum length of the password.
      /// </summary>
      /// <value>The minimum length of the password.</value>
      int MinimumPasswordLength
      {
         get;
         set;
      }

      /// <summary>
      /// Gets or sets the maximum length of the password.
      /// </summary>
      /// <value>The maximum length of the password.</value>
      int MaximumPasswordLength
      {
         get;
         set;
      }

      /// <summary>
      /// If policy requires mixed case
      /// </summary>
      /// <value>true if policy needs mixed case</value>
      bool RequireMixedCase
      {
         get;
         set;
      }

      /// <summary>
      /// If policy needs digits
      /// </summary>
      /// <value>true if policy needs digits.</value>
      bool RequireDigits
      {
         get;
         set;
      }

      /// <summary>
      /// If policy needs special characters
      /// </summary>
      /// <value>
      /// true if require special characters are needed
      /// </value>
      bool RequireSpecialCharacters
      {
         get;
         set;
      }

      /// <summary>
      /// Indicates if the username needs to be additionally
      /// supplied to verify the password complexity against
      /// </summary>
      /// <value>
      /// true require username to check password against
      /// </value>
      bool RequireUsernameToCheckPasswordAgainst
      {
         get;
         set;
      }

      /// <summary>
      /// Gets or sets the maximum count of characters
      /// in a sequence
      /// </summary>
      /// <value>The maximum count of characters
      /// in a sequence.</value>
      int MaximumCharacterSequenceCount
      {
         get;
         set;
      }

      /// <summary>
      /// The duration of the lockout in minutes.
      /// </summary>
      /// <remarks>
      /// This security setting determines the number of
      /// minutes a locked-out account remains locked
      /// out before automatically becoming unlocked.
      /// The available range is from 0 minutes through
      /// 99,999 minutes.
      /// If you set the account lockout duration to less
      /// than zero, the account will be locked out until an
      /// administrator explicitly unlocks it. If an account
      /// lockout threshold is defined, the account lockout
      /// duration must be greater than or equal to
      /// the reset time.
      /// </remarks>
      /// <value>The duration of the lockout.</value>
      int LockoutDuration
      {
         get;
         set;
      }

      /// <summary>
      /// Gets or sets the lockout threshold.
      /// </summary>
      /// <remarks>
      /// This security setting determines the number of
      /// failed logon attempts that causes a user
      /// account to be locked out. A locked-out
      /// account cannot be used until it is reset
      /// by an administrator or until the
      /// lockout duration for the account has expired. You
      /// can set a value between 0 and 999 failed
      /// logon attempts. If you set the value to 0,
      /// the account will never be locked out.
      /// </remarks>
      /// <value>The lockout threshold.</value>
      int LockoutThreshold
      {
         get;
         set;
      }

      /// <summary>
      /// Reset account lockout after X minutes
      /// </summary>
      /// <remarks>
      /// This security setting determines the number of
      /// minutes that must elapse after a failed logon
      /// attempt before the failed logon attempt
      /// counter is reset to 0 bad logon attempts.
      /// The available range is 1 minute to
      /// 99,999 minutes.
      /// </remarks>
      /// <value>The duration of the lockout.</value>
      int LockoutResetInMinutes
      {
         get;
         set;
      }
   }

You can see this password policy template extends the initial outline to not only provide guidance for the number of entropy bits, but allows for the policy to cover the lock out strategy in the case of incorrect password handling and password expiry approaches. If you look at the source code you will also see the options that are available, but for the sake of this article we are trying to keep on point :-)

So on to the actual strength testing, this oddly is rather simple at the end of the day. We are going to use an Interface definition (IPassword) for the Password processor (makes testing & mocking easier) so we can actually have multiple implementations (think MEF!).  Now the actual implementation.

  1. Check for sequences using various lookup tables to determine if any sequences exist. If a sequence length is detected, and is longer than allowed the password fails the policy. The tables include:
    • Alphabetic + numeric sequence
    • QWERTY US Keyboard
    • QWERTY UK Keyboard
    • AZERTY Keyboard
  2. Perform simple DecodeEliteEncoding then perform a simple hardcoded dictionary match of well known super common passwords
  3. If supplied (and if required) compare password elements to the user name

The end implementation is still fairly simple and it would be fairly easy to improve on this implementation.  The most obvious ones are to support a custom dictionary and add more custom keyboard sequences. Other extensions would be to store the passwords and become a real password token service. We can leave it up to the reader to provide an implementation of IPassword to call the Google password rating service rather than the above implementation:

https://www.google.com/accounts/RatePassword?Passwd=csharphacker

All good stuff! I hope this helps (and the source code) people provide a better approach to helping strengthen passwords.

[Download source code here]

The linked source code is liable to change over time so check back often. The source code uses the Microsoft testing framework and currently has 100% code coverage! Although I don’t think 100% is all that people think it is.

Finally the goal is to make everything a more secure place – and in reality the best approach is to use a strong memorable password in conjunction with a hardware token that changes every minute.

As always feedback is welcome!

Gareth