Blog »

Launching process programmatically: system vs CreateProcess vs ShellExecute

Sat
15
Apr 2023

Today I went on a quest to investigate various ways in which we can launch a process (an EXE file) programmatically, while programming in C++ using Windows. I tested 3 different functions: system, CreateProcess, ShellExecute. I focused on ways to specify a path to the executable file – not on passing parameters and not on capturing standard input/output of the subprocess (which I plan to investigate next and I did). All examples below launch a subprocess and wait until it completes before continuing. They all make the subprocess inheriting the console, so if both main process and the subprocess are console programs, their output will go to the single console of the main process.

But first things first: To understand this article, please recall that in operating systems we commonly use, no matter if Windows or Linux, every executable file launched as a process has several parameters:

Paths in the file system can be absolute (in case of Windows it usually means they start with drive letter, like “C:\Dir1\Text.exe”) or relative.

Startup directory is often the same as the directory where the executable file is located, but it doesn’t need to be. Many methods of process launching offer an explicit parameter for it. We won’t use it in the code samples below, but you can also achieve this manually from system console. For example, following console command uses a relative path to launch an executable located in “C:\Dir2\Test.exe”, while current directory of the process will be the same as current directory of the console: “C:\Dir1”:

C:\Dir1>..\Dir2\Test.exe

Method 1: Function system from standard C library (required header: <stdlib.h> or <cstdlib> in C++) is the simplest, most primitive one. It just takes a single string as parameter. An advantage of it is that you can launch any console command with it, also built-in commands (like “echo”), not only EXE files. It is also portable between different operating systems.

#include <cstdlib>

int main()
{
    char path[MAX_PATH];
    strcpy_s(path, "Test.exe");

    system(path);
}

Waiting for the command to finish is the default behavior of this function and so is inheriting the console, so that messages printed to the standard output by “Test.exe” will go to the same console as our host application.

path can always be absolute or relative. For each of the 4 methods described in this article, I found answers to following questions:

  1. Does it work with file name alone without an extension, like "Test"? In case of this function: Yes, it does.
  2. When only file name (relative path) is specified, like "Test.exe", where is the function able to locate the executable file?
    1. In the same directory where host EXE is located? No.
    2. In the current directory? Yes.
    3. In one of the directories passed though PATH environmental variable? Yes.
  3. When the path contains spaces, like "C:\My Program\Test.exe", how to escape it properly?
    1. Does it work as-is without any escaping, like strcpy_s(path, "C:\\My Program\\Test.exe");? No. (Note the double backslash \\ is for escaping in C++, so that string will actually contain single backslashes. You can also use forward slashes / in Windows – they work with all methods described in this article and they don’t need to be escaped in C++ code.)
    2. Does it work when entire path is enclosed with quotes, like strcpy_s(path, "\"C:\\My Program\\Test.exe\"");? Yes.
    3. Does it work when spaces are escaped with character ^, like strcpy_s(path, "C:\\My^ Program\\Test.exe");? Yes! (However strange it looks, this is the character used as an escape sequence in Windows shell!)

Method 2: Function CreateProcess from WinAPI (required header: <Windows.h>) is likely the most native and most feature-rich option. Numerous parameters passed to the function and accompanying structures allow to control the new subprocess in various ways, including getting and using its process handle or capturing its standard input/output. Here, for simplicity, I replicate the behavior of system function from method 1 – I make it inherit the console by passing parameter bInheritHandles = TRUE and wait until it completes by calling WaitForSingleObject on the process handle. Process handle and main thread handle also need to closed to avoid resource leak.

STARTUPINFO startupInfo = { sizeof(STARTUPINFO) };
PROCESS_INFORMATION processInfo = {};
BOOL success = CreateProcess(
    path, // lpApplicationName
    NULL, // lpCommandLine
    NULL, // lpProcessAttributes
    NULL, // lpThreadAttributes
    TRUE, // bInheritHandles
    0, // dwCreationFlags
    NULL, // lpEnvironment
    NULL, // lpCurrentDirectory
    &startupInfo,
    &processInfo);
assert(success);
WaitForSingleObject(processInfo.hProcess, INFINITE);
CloseHandle(processInfo.hThread);
CloseHandle(processInfo.hProcess);

There are actually 2 ways to pass executable file path to CreateProcess. Code above shows the first way – using lpApplicationName parameter, which is intended for just application name, while command line parameters are passed via next argument. Note this is different from system function, which accepts one string with everything. Using the method shown above:

  1. Does it work with file name alone without an extension, like "Test"? No.
  2. When only file name is specified, like "Test.exe", where is the function able to locate the executable file?
    1. In the same directory where host EXE is located? No.
    2. In the current directory? Yes.
    3. In one of the directories passed though PATH environmental variable? No!
  3. When the path contains spaces, how to escape it properly?
    1. Does it work as-is without any escaping, like "C:\\My Program\\Test.exe"? Yes – likely because this parameter is intended exclusively for executable file path.
    2. Does it work when entire path is enclosed with quotes, like "\"C:\\My Program\\Test.exe\""? No.
    3. Does it work when spaces are escaped with character ^, like "C:\\My^ Program\\Test.exe"? No.

Method 3: Function CreateProcess, but this time passing executable file path as lpCommandLine parameter, while leaving lpApplicationName set to NULL. This is also a valid use case and it behaves differently – more like launching a console command than starting a specific EXE file.

STARTUPINFO startupInfo = { sizeof(STARTUPINFO) };
PROCESS_INFORMATION processInfo = {};
BOOL success = CreateProcess(
    NULL, // lpApplicationName  <- !!!
    path, // lpCommandLine      <- !!!
    NULL, // lpProcessAttributes
    NULL, // lpThreadAttributes
    TRUE, // bInheritHandles
    0, // dwCreationFlags
    NULL, // lpEnvironment
    NULL, // lpCurrentDirectory
    &startupInfo,
    &processInfo);
assert(success);
WaitForSingleObject(processInfo.hProcess, INFINITE);
CloseHandle(processInfo.hThread);
CloseHandle(processInfo.hProcess);
  1. Does it work with file name alone without an extension, like "Test"? Yes!
  2. When only file name is specified, like "Test.exe", where is the function able to locate the executable file?
    1. In the same directory where host EXE is located? Yes!
    2. In the current directory? Yes.
    3. In one of the directories passed though PATH environmental variable? Yes!
  3. When the path contains spaces, how to escape it properly?
    1. Does it work as-is without any escaping, like "C:\\My Program\\Test.exe"? No!
    2. Does it work when entire path is enclosed with quotes, like "\"C:\\My Program\\Test.exe\""? Yes.
    3. Does it work when spaces are escaped with character ^, like "C:\\My^ Program\\Test.exe"? No!

Method 4: Function ShellExecuteEx (or legacy ShellExecute) which is also part of WinAPI, but coming from header <shellapi.h>. It requires COM to be initialized with CoInitializeEx. It can be used not only to start processes from EXE files, but also to open any types of files (TXT or DOCX documents, JPEG images etc.) with their associated programs, as if the user double-clicked on such file or right-clicked and selected one of the available “verbs”, like “Edit” or “Print”. But for this article, let’s focus on launching executable files. To replicate the same behavior as in previous methods, I pass SEE_MASK_NO_CONSOLE to inherit console and SEE_MASK_NOCLOSEPROCESS to retrieve process handle to be able to wait for it.

CoInitializeEx(NULL, COINIT_APARTMENTTHREADED | COINIT_DISABLE_OLE1DDE);
SHELLEXECUTEINFO shellExecuteInfo = {
    .cbSize = sizeof(SHELLEXECUTEINFO),
    .fMask = SEE_MASK_NOCLOSEPROCESS | SEE_MASK_NO_CONSOLE,
    .lpFile = path,
    .nShow = SW_SHOWNORMAL
};
BOOL success = ShellExecuteEx(&shellExecuteInfo);
assert(success);
WaitForSingleObject(shellExecuteInfo.hProcess, INFINITE);
CloseHandle(shellExecuteInfo.hProcess);

This method behaves in the following way:

  1. Does it work with file name alone without an extension, like "Test"? Yes.
  2. When only file name is specified, like "Test.exe", where is the function able to locate the executable file?
    1. In the same directory where host EXE is located? No!
    2. In the current directory? Yes.
    3. In one of the directories passed though PATH environmental variable? Yes.
  3. When the path contains spaces, how to escape it properly?
    1. Does it work as-is without any escaping, like "C:\\My Program\\Test.exe"? Yes.
    2. Does it work when entire path is enclosed with quotes, like "\"C:\\My Program\\Test.exe\""? Yes.
    3. Does it work when spaces are escaped with character ^, like "C:\\My^ Program\\Test.exe"? No.

To summarize, let’s see all the results in a table:

  system() CreateProcess()
lpApplicationName		 CreateProcess()
lpCommandLine		 ShellExecuteEx()
Works without extension? "Test" YesNoYesYes
Searching dir of the host app? NoNoYesNo
Searching current dir? YesYesYesYes
Searching PATH env var? YesNoYesYes
Path with spaces unescaped: My Program\Test.exe NoYesNoYes
Path with spaces enclosed with quotes: "My Program\Test.exe" YesNoYesYes
Spaces escaped with ^: My^ Program\Test.exe YesNoNoNo

I did my tests using Windows 10, Version 22H2 (OS Build 19045.2846) and Visual Studio 2022 17.5.3. Although unlikely, it is not impossible that these results may change on another version of the operating system or C++ compiler and standard library implementation.

Comments | #windows #c++ #winapi Share

Comments

[Download] [Dropbox] [pub] [Mirror] [Privacy policy]
Copyright © 2004-2024