Windows Server 2012 SMB 3.0 Invalid Signature Error with SMB 2 Storage

Overview

During a recent migrating to a new backup solution using Commvault on Windows Server 2012, we ran into an error that prevented us from writing backup data to a Dell Compellent running FluidFS version 3. What made it difficult to troubleshoot was that we were able to write the initial directory structure to the FluidFS volumes, however, the connections to the storage would then be immediately severed by an invalid signature error.

Seeing that the directory structure was being written, we knew it wasn’t a permissions issue. During the troubleshooting phase manually copied files to the NAS without any problems; we mounted a separate SMB share hosted by a Windows server, and Simpana had no trouble writing to it.

The cause of our problems turned out to be due to new features introduced in Windows Server 2012 and SMB 3.0. The new SMB has a feature called Secure Negotiate. According to Microsoft: this feature depends upon the correct signing of error responses by all SMBv2 servers, including servers that support only protocol versions 2.0 and 2.1. Some third-party file servers do not return a signed error response. Therefore, the connection fails. The feature is enabled by default and must be disabled when using NAS storage that doesn’t support signed error responses, like Dell’s FluidFS on Compellent.

The solution, unless your storage vendor releases patch to support the feature, is to disable the feature in Windows. If you are interested in reading the published knowledgebase article from Microsoft, it can be found in Microsoft’s KB 2686098.

What is Secure Negotiate

This new feature introduced in SMB 3.0 is used to prevent man-in-the-middle attacks. Windows 8 and Windows Server 2012 clients connecting to SMB sharee expect a valid signature response back from the server, to valide transmitted data hasn’t been manipulated.

This meant that our Windows Server 2012 box was expecting a valid signature back from our FluidFS SMB share hosted on Dell Compellent. Because, at the time of this writing, it secure signatures are not supported by FluidFS, the client would kill the connection.

Disable Secure Negotiate

There are a few ways to disable this feature. Which one you choose depends on your environment and your comfort level with administrating the operating system.

PowerShell

This is the quickest method to disable the Security Signature. If only a few servers are affected, you can use this method until a patch is released by your storage vendor.

  1. Open a new PowerShell window.
  2. Verify that Secure Negotiate is enabled by running the command Get-SmbClientConfiguration. Look for the setting named EnableSecuritySignature. By default it is enabled.
  3. To disable it, run the cmdlet Set-SmbClientConfiguration.
    Set-SmbClientConfiguration -EnableSecuritySignature $false
  4. You can verify the that the command was success by running Get-SmbClientConfiguration and looking at its output. An example is shown below.
    Windows PowerShell
    Copyright (C) 2013 Microsoft Corporation. All rights reserved.
    
    PS C:\Users\Administrator> Get-SmbClientConfiguration
    
    ConnectionCountPerRssNetworkInterface : 4
    DirectoryCacheEntriesMax              : 16
    DirectoryCacheEntrySizeMax            : 65536
    DirectoryCacheLifetime                : 10
    EnableBandwidthThrottling             : True
    EnableByteRangeLockingOnReadOnlyFiles : True
    EnableLargeMtu                        : True
    EnableMultiChannel                    : True
    DormantFileLimit                      : 1023
    EnableSecuritySignature               : False
    ExtendedSessionTimeout                : 1000
    FileInfoCacheEntriesMax               : 64
    FileInfoCacheLifetime                 : 10
    FileNotFoundCacheEntriesMax           : 128
    FileNotFoundCacheLifetime             : 5
    KeepConn                              : 600
    MaxCmds                               : 50
    MaximumConnectionCountPerServer       : 32
    OplocksDisabled                       : False
    RequireSecuritySignature              : False
    SessionTimeout                        : 60
    UseOpportunisticLocking               : True
    WindowSizeThreshold                   : 1

Local Security Policy

Modifying the local security policy is another method of disabling the SMB 3.0 security signature feature. This is ideal in small environments where maybe only one or two servers is affected. This policy can then be saved an applied to new servers as they’re deployed. Hopefully, this won’t be necessary if your storage vendor releases a patch.

  1. Launch the Server Manager console.
  2. Click on the Tools menu and then select Local Security Policy.
  3. Expand Security Settings.
  4. Expand Local Policies.
  5. Expand Security Options.
  6. Look for a policy named Microsoft network client: Digitally sign communications (if server agrees).

    Local Security Policy - Microsoft network client: Digitally sign communications.
    Local Security Policy – Microsoft network client: Digitally sign communications.
  7. Right-click the policy and then click Properties.
  8. Select the Disable radio, and then click OK.
  9. Close the Local Security Policy console.

Group Policy

Group policies is recommend in large environments. It allows you to quickly apply the fix to a large number of servers. When a patch is finally released by your storage vendor, you can easily enable the Security Signature used by SMB 3.0.

  1. Launch the Group Policy Management console.
  2. Create a new Group Policy Object (GPO).
  3. Right-click the newly created GPO and then click Edit.
  4. Expand Computer Configuration.
  5. Expand Windows Settings.
  6. Expand Security Settings.
  7. Expand Local Policies.
  8. Browse the list of policies found in Security Options and look for Microsoft network client: Digitally sign communications (if server agrees).

    Group Policy Management Editor
    Group Policy Management Editor
  9. Right-click the policy and then click Properties.
  10. Check the Define this policy setting checkbox and then select Disabled.

    Microsoft network client: Digitally sign communications (if server agrees) GPO.
    Microsoft network client: Digitally sign communications (if server agrees) GPO.
  11. Click OK to close the GPO’s properties dialog box.
  12. Close the Group Policy Management Editor console.
  13. Assign the GPO to the appropriate servers or OU’s.
  14. Run the gpupdate command on affected servers to apply the new group policy.