FraserOfSmeg FraserOfSmeg - 2 months ago 19
Vb.net Question

Why do some applications not accept some sendkeys at some times

This is an issue I've ran into before, but I've always given up solving the problem and worked out a work around. Not today (hopefully).

I'm trying to make a bot for the classic Doom II. I want my bot to have access to the main menu which is accessed via the escape key. Naturally I tried:

sendkeys.send("{ESC}")


No luck. But then something weird happened. I accidently ran the code when I was already on the menu... and it closed the menu (which is normal if you press escape on the menu). So clearly Doom II listens to Sendkeys.

I've since tried sendinput, postmessage, and simulateinput. None have worked (they all have the same behaviour as described with sendkeys).

It would be great if someone could ride in on a white horse and give me code to get around this issue, but outside of that can any one simply explain this behaviour to me?

Answer

It seems that Zandronum does not accept virtual keys to be sent to it when the game is running (not paused). I'm not sure but it seems that virtual keys might actually be window messages, like Andrew Morton said (or they're at least something similar...). The workaround to this was to send a hardware scan code instead of a virtual key code.

A hardware scan code appears to be the code sent by the actual keyboard when pressing a key, while a virtual key code is the key which the system interprets from the scan code (reference).

So I managed to send keystrokes to Zandronum (both fullscreen and windowed) using a few WinAPI functions:

  • SendInput() which is used to send the actual keyboard input.
  • MapVirtualKeyEx() which is used to convert key codes to scan codes, or vice versa.
  • GetKeyboardLayout() which is used to get the user's current keyboard layout (I, for example, have a Swedish keyboard).

By using the below helper class (or more correctly: wrapper) that I built you may now send keystrokes (hardware or not) in a simple manner, with a larger variety of keys than what SendKeys.Send() includes. You may use any key in the System.Windows.Forms.Keys enumeration.

This was tested with Zandronum and works completely:

InputHelper.PressKey(Keys.Escape, True) 'True = Send key as hardware scan code.

InputHelper.vb:

Imports System.Runtime.InteropServices

Public NotInheritable Class InputHelper
    Private Sub New()
    End Sub

#Region "Methods"
#Region "PressKey()"
    ''' <summary>
    ''' Virtually presses a key.
    ''' </summary>
    ''' <param name="Key">The key to press.</param>
    ''' <param name="HardwareKey">Whether or not to press the key using its hardware scan code.</param>
    ''' <remarks></remarks>
    Public Shared Sub PressKey(ByVal Key As Keys, Optional ByVal HardwareKey As Boolean = False)
        If HardwareKey = False Then
            InputHelper.SetKeyState(Key, False)
            InputHelper.SetKeyState(Key, True)
        Else
            InputHelper.SetHardwareKeyState(Key, False)
            InputHelper.SetHardwareKeyState(Key, True)
        End If
    End Sub
#End Region

#Region "SetKeyState()"
    ''' <summary>
    ''' Virtually sends a key event.
    ''' </summary>
    ''' <param name="Key">The key of the event to send.</param>
    ''' <param name="KeyUp">Whether to push down or release the key.</param>
    ''' <remarks></remarks>
    Public Shared Sub SetKeyState(ByVal Key As Keys, ByVal KeyUp As Boolean)
        Key = ReplaceBadKeys(Key)

        Dim KeyboardInput As New KEYBDINPUT With {
            .wVk = Key,
            .wScan = 0,
            .time = 0,
            .dwFlags = If(KeyUp, KEYEVENTF.KEYUP, 0),
            .dwExtraInfo = IntPtr.Zero
        }

        Dim Union As New INPUTUNION With {.ki = KeyboardInput}
        Dim Input As New INPUT With {
            .type = INPUTTYPE.KEYBOARD,
            .U = Union
        }

        SendInput(1, New INPUT() {Input}, Marshal.SizeOf(GetType(INPUT)))
    End Sub
#End Region

#Region "SetHardwareKeyState()"
    ''' <summary>
    ''' Virtually sends a key event using the key's scan code.
    ''' </summary>
    ''' <param name="Key">The key of the event to send.</param>
    ''' <param name="KeyUp">Whether to push down or release the key.</param>
    ''' <remarks></remarks>
    Public Shared Sub SetHardwareKeyState(ByVal Key As Keys, ByVal KeyUp As Boolean)
        Key = ReplaceBadKeys(Key)

        Dim KeyboardInput As New KEYBDINPUT With {
            .wVk = 0,
            .wScan = MapVirtualKeyEx(CUInt(Key), 0, GetKeyboardLayout(0)),
            .time = 0,
            .dwFlags = KEYEVENTF.SCANCODE Or If(KeyUp, KEYEVENTF.KEYUP, 0),
            .dwExtraInfo = IntPtr.Zero
        }

        Dim Union As New INPUTUNION With {.ki = KeyboardInput}
        Dim Input As New INPUT With {
            .type = INPUTTYPE.KEYBOARD,
            .U = Union
        }

        SendInput(1, New INPUT() {Input}, Marshal.SizeOf(GetType(INPUT)))
    End Sub
#End Region

#Region "ReplaceBadKeys()"
    ''' <summary>
    ''' Replaces bad keys with their corresponding VK_* value.
    ''' </summary>
    ''' <remarks></remarks>
    Private Shared Function ReplaceBadKeys(ByVal Key As Keys) As Keys
        Dim ReturnValue As Keys = Key

        If ReturnValue.HasFlag(Keys.Control) Then
            ReturnValue = (ReturnValue And Not Keys.Control) Or Keys.ControlKey 'Replace Keys.Control with Keys.ControlKey.
        End If

        If ReturnValue.HasFlag(Keys.Shift) Then
            ReturnValue = (ReturnValue And Not Keys.Shift) Or Keys.ShiftKey 'Replace Keys.Shift with Keys.ShiftKey.
        End If

        If ReturnValue.HasFlag(Keys.Alt) Then
            ReturnValue = (ReturnValue And Not Keys.Alt) Or Keys.Menu 'Replace Keys.Alt with Keys.Menu.
        End If

        Return ReturnValue
    End Function
#End Region
#End Region

#Region "WinAPI P/Invokes"
    <DllImport("user32.dll", SetLastError:=True)>
    Private Shared Function SendInput(ByVal nInputs As UInteger, <MarshalAs(UnmanagedType.LPArray)> ByVal pInputs() As INPUT, ByVal cbSize As Integer) As UInteger
    End Function

    <DllImport("user32.dll")> _
    Private Shared Function MapVirtualKeyEx(uCode As UInteger, uMapType As UInteger, dwhkl As IntPtr) As UInteger
    End Function

    <DllImport("user32.dll")> _
    Private Shared Function GetKeyboardLayout(idThread As UInteger) As IntPtr
    End Function

#Region "Enumerations"
    Private Enum INPUTTYPE As UInteger
        MOUSE = 0
        KEYBOARD = 1
        HARDWARE = 2
    End Enum

    <Flags()> _
    Private Enum KEYEVENTF As UInteger
        EXTENDEDKEY = &H1
        KEYUP = &H2
        SCANCODE = &H8
        UNICODE = &H4
    End Enum
#End Region

#Region "Structures"
    <StructLayout(LayoutKind.Explicit)> _
    Private Structure INPUTUNION
        <FieldOffset(0)> Public mi As MOUSEINPUT
        <FieldOffset(0)> Public ki As KEYBDINPUT
        <FieldOffset(0)> Public hi As HARDWAREINPUT
    End Structure

    Private Structure INPUT
        Public type As Integer
        Public U As INPUTUNION
    End Structure

    Private Structure MOUSEINPUT
        Public dx As Integer
        Public dy As Integer
        Public mouseData As Integer
        Public dwFlags As Integer
        Public time As Integer
        Public dwExtraInfo As IntPtr
    End Structure

    Private Structure KEYBDINPUT
        Public wVk As UShort
        Public wScan As Short
        Public dwFlags As UInteger
        Public time As Integer
        Public dwExtraInfo As IntPtr
    End Structure

    Private Structure HARDWAREINPUT
        Public uMsg As Integer
        Public wParamL As Short
        Public wParamH As Short
    End Structure
#End Region
#End Region
End Class

Just for fun, I also managed to find a list of scan codes on the MSDN: https://msdn.microsoft.com/en-us/library/aa299374(v=vs.60).aspx


Since I'm a Doom fan myself and familiar with how it works, perhaps you should (per your old question) also make sure that you have selected New Game in the menu before you make it press enter?

Zandronum is aware of the names of the menu items, so you just have to give it the first letter and it will jump to the item starting with it:

InputHelper.PressKey(Keys.Escape, True) 'Open the menu.
System.Threading.Thread.Sleep(100)      'Small delay to let the menu open.
InputHelper.PressKey(Keys.N, True)      'Jump to the "New Game" menu item.
InputHelper.PressKey(Keys.Enter, True)  'Go into the "New Game" menu.
InputHelper.PressKey(Keys.Enter, True)  'Start a new game.

I've tested the above code in-game, running in fullscreen mode. Works like a charm.

Comments