C++ For C# Developers: Part 41 – System Integration Library
A programming language without access to the underlying system is of little use. Even a “Hello, world!” program requires the OS to output that message. Today we’ll start looking at the system access that the Standard Library provides. We’ll see how to access the file system, so-called “smart” pointers, and check the time using various system clocks.
Table of Contents
- Part 1: Introduction
- Part 2: Primitive Types and Literals
- Part 3: Variables and Initialization
- Part 4: Functions
- Part 5: Build Model
- Part 6: Control Flow
- Part 7: Pointers, Arrays, and Strings
- Part 8: References
- Part 9: Enumerations
- Part 10: Struct Basics
- Part 11: Struct Functions
- Part 12: Constructors and Destructors
- Part 13: Initialization
- Part 14: Inheritance
- Part 15: Struct and Class Permissions
- Part 16: Struct and Class Wrap-up
- Part 17: Namespaces
- Part 18: Exceptions
- Part 19: Dynamic Allocation
- Part 20: Implicit Type Conversion
- Part 21: Casting and RTTI
- Part 22: Lambdas
- Part 23: Compile-Time Programming
- Part 24: Preprocessor
- Part 25: Intro to Templates
- Part 26: Template Parameters
- Part 27: Template Deduction and Specialization
- Part 28: Variadic Templates
- Part 29: Template Constraints
- Part 30: Type Aliases
- Part 31: Deconstructing and Attributes
- Part 32: Thread-Local Storage and Volatile
- Part 33: Alignment, Assembly, and Language Linkage
- Part 34: Fold Expressions and Elaborated Type Specifiers
- Part 35: Modules, The New Build Model
- Part 36: Coroutines
- Part 37: Missing Language Features
- Part 38: C Standard Library
- Part 39: Language Support Library
- Part 40: Utilities Library
- Part 41: System Integration Library
- Part 42: Numbers Library
- Part 43: Threading Library
- Part 44: Strings Library
- Part 45: Array Containers Library
- Part 46: Other Containers Library
- Part 47: Containers Library Wrapup
- Part 48: Algorithms Library
- Part 49: Ranges and Parallel Algorithms
- Part 50: I/O Library
- Part 51: Missing Library Features
- Part 52: Idioms and Best Practices
- Part 53: Conclusion
Chrono
The <chrono>
header provides time- and date-related functionality. It’s design approach is to apply the strong class types system to individual units of time. The C# equivalent is the DateTime
and Stopwatch
classes as well as the TimeSpan
struct. Other systems prefer a less strongly-typed approach. This includes Unity’s usage of a float
representing seconds.
Let’s start by looking at the different “clock” classes. Each of these represents a different way of checking the time:
#include <chrono> // Everything in <chrono> is in the std::chrono namespace using namespace std::chrono; // Check the "wall clock" which represents real-world time // It specializes the time_point class template to hold a point in time // This is the elapsed time since midnight UTC on January 1, 1970 time_point<system_clock> sys = system_clock::now(); // Convert to a std::time_t integer representing seconds DebugLog(system_clock::to_time_t(sys)); // Maybe 1613931944 // Check the "monotonic clock" which always moves forward // Never moves backwards when the system time changes // Never moves backwards for daylight savings time // Good for measuring intervals time_point<steady_clock> steady = steady_clock::now(); // Check the "high resolution clock" which provides maximum precision // Not guaranteed to be monotonic time_point<high_resolution_clock> highRes = high_resolution_clock::now();
C++20 provides more clock types:
#include <chrono> using namespace std::chrono; // Represents real-world time in UTC time_point<utc_clock> utc = utc_clock::now(); // Represents International Atomic Time time_point<tai_clock> tai = tai_clock::now(); // Represents GPS time time_point<gps_clock> gps = gps_clock::now(); // Represents file system time time_point<file_clock> file = file_clock::now();
Besides time_point
, we can also represent durations using the duration
class. These result from subtracting two time_point
objects:
#include <chrono> using namespace std::chrono; // Check the monotonic time before doing something expensive time_point<steady_clock> before = steady_clock::now(); // Do something expensive volatile float f = 3.14f; for (volatile int i = 0; i < 100000000; ++i) { f *= f; } // Check the monotonic time afterward time_point<steady_clock> after = steady_clock::now(); // Subtract time_point objects to get a duration // Specify that we want milliseconds as a double duration<double, std::milli> elapsed = after - before; // Extract the primitive value from a duration double elapsedMs = elapsed.count(); DebugLog(elapsedMs); // Maybe 319.781 // Specify that we want milliseconds as an integer that's precise enough // Not a real cast, just a function with "cast" in the name microseconds elapsedMsInt = duration_cast<microseconds>(after - before); DebugLog(elapsedMsInt.count()); // Maybe 319781
microseconds
is one of the type aliases provided for common durations:
#include <chrono> using namespace std::chrono; nanoseconds cpuCycleAt500MHz{ 2 }; microseconds blinkOfEye{ 350000 }; milliseconds frame{ 33 }; seconds countdown{ 5 }; minutes matchLength{ 10 }; hours day{ 24 }; // Starting in C++20... days weekend{ 2 }; weeks fortnight{ 2 }; months quarter{ 4 }; years decade{ 10 };
We also have some user-defined literals representing common durations in the std::literals::chrono_literals
namespace:
#include <chrono> using namespace std::chrono; using namespace std::literals::chrono_literals; hours day = 24h; minutes matchLength = 10min; seconds countdown = 5s; milliseconds frame = 33ms; microseconds blinkOfEye = 350000us; nanoseconds cpuCycleAt500MHz = 2ns;
The duration
class template is able to implicitly convert to other instantiations of the template. This means the compiler will automatically insert the proper unit conversions so we can’t accidentally use the wrong time units:
#include <chrono> using namespace std::chrono; using namespace std::literals::chrono_literals; // Function that delays for a number of nanoseconds template <typename TCallback> void DelayThenCall(nanoseconds delay, TCallback callback) { // Spin until enough time has passed auto before = steady_clock::now(); while (duration_cast<nanoseconds>(steady_clock::now() - before) < delay) { } callback(); } // Print the time before DebugLog(system_clock::to_time_t(system_clock::now())); // Maybe 1613934909 // Pass milliseconds instead of nanoseconds // Compiler automatically does the conversion from milliseconds to nanoseconds DelayThenCall( 1234ms, [&] { // Print the time after time_t sec = system_clock::to_time_t(system_clock::now()); DebugLog(sec); // Maybe 1613934911 });
Filesystem
Starting in C++17, we have <filesystem>
to deal with the file system. This is distinct from reading and writing to individual files. Instead, it provides functionality to deal with paths, directories, links, and so forth. The difference is similar to the File
class in C# that deals with the contents of files while the Path
and Directory
classes deal with the file system.
Let’s start with the path
class which is similar to Path
in C#:
#include <filesystem> // Everything in <filesystem> is in the std::filesystem namespace using namespace std::filesystem; // Get a path to a directory path dir{ "/path/to" }; // Extract the string from the path DebugLog(dir.string()); // /path/to // Concatenate with the overloaded / operator path file = dir / path{ "file.dat" }; DebugLog(file.string()); // /path/to/file.dat // Get parts of the path DebugLog(file.filename()); // file.dat DebugLog(file.extension()); // .dat DebugLog(file.parent_path()); // /path/to DebugLog(file.root_directory()); // / // Change just the file name part of the path file.replace_filename("otherFile.dat"); DebugLog(file.string()); // /path/to/otherFile.dat // Change just the extension of the file name part of the path file.replace_extension(".tex"); DebugLog(file.string()); // /path/to/otherFile.tex // Remove the file name to get the directory it's in file.remove_filename(); DebugLog(file.string()); // /path/to/
Once we have a path
we can get information about it with several namespace-level functions:
#include <filesystem> std::filesystem::path p{ "/path/to/file" }; DebugLog(std::filesystem::is_directory(p)); // false DebugLog(std::filesystem::is_regular_file(p)); // true DebugLog(std::filesystem::is_symlink(p)); // Maybe false DebugLog(std::filesystem::is_empty(p)); // Maybe false
There are also namespace-level functions to deal with path
objects:
#include <filesystem> using namespace std::filesystem; // Get the current working directory path curPath = std::filesystem::current_path(); DebugLog(curPath.string()); // Maybe /myCurrentPath // Get an absolute path from the current working directory path relPath{ "subDir/file.dat" }; path absPath = std::filesystem::absolute(relPath); DebugLog(absPath.string()); // Maybe /myCurrentPath/subDir/file.dat // Get a relative path to another directory path otherRelPath = std::filesystem::relative( path{ "/otherDir/subDir/file.dat" }, // Absolute path path{ "/otherDir" }); // Base directory to be relative to DebugLog(otherRelPath.string()); // subDir/file.dat // Get the system's temporary directory path tempPath = std::filesystem::temp_directory_path(); DebugLog(tempPath.string()); // Maybe /path/to/temp/dir // Check if paths refer to the same place DebugLog(std::filesystem::equivalent(relPath, absPath)); // Maybe true
We have quite a few ways to query the file system:
#include <filesystem> namespace fs = std::filesystem; // Check if a file exists fs::path p{ "/path/to/file.dat" }; DebugLog(fs::exists(p)); // Maybe false // Get a file's size DebugLog(fs::file_size(p)); // Maybe 123 // Check how many hard links refer to the file DebugLog(fs::hard_link_count(p)); // Maybe 1 // Check the last time the file was written to // Returns an alias to a time_point<file_clock> fs::file_time_type time = fs::last_write_time(p); DebugLog(time.time_since_epoch().count()); // Maybe 132477133770000000 // Check free space fs::space_info si = fs::space(p); DebugLog(si.capacity); // Maybe 494206447616 DebugLog(si.free); // Maybe 45688299520 DebugLog(si.available); // Maybe 45688299520 // Check the status of a file fs::file_status status = fs::status(p); DebugLog(status.type() == fs::file_type::regular); // Maybe true fs::perms perms = status.permissions(); bool canOwnerWrite = (perms & fs::perms::owner_write) != fs::perms::none; DebugLog(canOwnerWrite); // Maybe true
Of course there are also a lot of functions to modify the file system too:
#include <filesystem> namespace fs = std::filesystem; // Set permissions fs::path p{ "/path/to/notempty.txt" }; fs::permissions(p, fs::perms::owner_read | fs::perms::owner_write); // Copy the file fs::copy( p, // Source fs::path{ "/path/to/notempty2.txt" }, // Destination fs::copy_options::overwrite_existing); // Options // Create a directory fs::create_directory(fs::path{ "/path/to/newDir" }); // Create a directory and all parent directories as necessary fs::create_directories(fs::path{ "/path/to/newDir\\newSubDir\\newSubSubDir" }); // Create a symlink fs::create_symlink(p, fs::path{ "/path/to/notempty_link.txt" }); // Delete a file or a directory fs::remove(fs::path{ "/path/to/newDir\\newSubDir\\newSubSubDir" }); // Delete a file or recursively delete a directory and all its contents fs::remove_all(fs::path{ "/path/to/newDir" });
New
The <new>
header mostly relates to the new
and delete
operators that we use for dynamic memory allocation. Since the memory manager isn’t customizable in C#, there’s no equivalent to this in that language. We’ve seen a bit of it already, such as the exception types thrown when new
fails:
#include <new> try { // Throws bad_alloc if the system can't allocate this much memory new char[0x7fffffffffffffff]; } catch (const std::bad_alloc& ex) { DebugLog(ex.what()); // Maybe bad alloc } try { // Work around compiler error for negative size volatile int i = -1; // Throws bad_array_new_length since length is negative new char[i]; } catch (const std::bad_array_new_length& ex) { DebugLog(ex.what()); // Maybe bad array new length }
We can also provide a function to customize what happens when new
fails to allocate:
#include <new> int callCount = 0; // Set the function to call when allocation fails // Provides an opportunity to get more memory or terminate the program std::set_new_handler([] { callCount++; }); // Get the function and call it std::get_new_handler()(); DebugLog(callCount); // 1 try { // If allocation fails, calls the "new handler" we set above new char[0x7fffffffffffffff]; } catch (const std::bad_alloc& ex) { DebugLog(ex.what()); // Maybe bad alloc } DebugLog(callCount); // 1 if allocation didn't fail, 2 if it did
Memory
Best practices guidelines for the “Modern C++” that’s existed since C++11 typically include a prohibition or minimization of the use of “raw pointers” and “naked new
and delete
.” Instead, they emphasize the usage of “smart pointers” that are classes that overload enough operators to behave like pointers. These types, provided by <memory>
, can avoid memory leaks by deallocating in their destructors. It’s similar to garbage collection in C# except that it’s done deterministically, synchronously, and usually a lot more efficiently.
For starters, let’s look at the std::unique_ptr
class template in <memory>
:
#include <memory> struct IntHolder { int Val; IntHolder(int val) : Val{ val } { DebugLog("ctor"); } ~IntHolder() { DebugLog("dtor"); } }; void Foo() { // Create a unique_ptr with make_unique // Internally calls "new IntHolder{123}" std::unique_ptr<IntHolder> p{ std::make_unique<IntHolder>(123) }; // ctor // Use it like an IntHolder* due to overloaded operators DebugLog(p->Val); // 123 // The unique_ptr destructor calls "delete P" where P is the IntHolder* } // dtor
To prevent there ever being more than one unique_ptr
managing the same object, its copy constructor and copy assignment operator are deleted so it cannot be copied. It can be “moved” though by passing an rvalue reference:
#include <memory> void TakeCopy(std::unique_ptr<IntHolder> p) { DebugLog(p->Val); } void TakeLvalueRef(std::unique_ptr<IntHolder>& p) { DebugLog(p->Val); } void TakeRvalueRef(std::unique_ptr<IntHolder>&& p) { DebugLog(p->Val); } void Foo() { std::unique_ptr<IntHolder> p{ std::make_unique<IntHolder>(123) }; // ctor // Compiler error: copy constructor is deleted TakeCopy(p); // OK TakeLvalueRef(p); // 123 // Compiler error: can't pass an lvalue reference to an rvalue reference TakeRvalueRef(p); // OK: casts lvalue reference to rvalue reference TakeRvalueRef(std::move(p)); // 123 } // dtor
A set of functions are included to make the class behave like a pointer and to manipulate what it points to:
#include <memory> void Foo() { std::unique_ptr<IntHolder> p{ std::make_unique<IntHolder>(123) }; // ctor // Convert to a bool if (p) { DebugLog("not null"); // gets printed } // Get the managed object. Be careful not to delete it! IntHolder* rawPtr = p.get(); DebugLog(rawPtr->Val); // 123 // Become null and return the managed object // It's now our responsibility to delete it! rawPtr = p.release(); // dtor DebugLog(rawPtr->Val); // 123 DebugLog((bool)p); // false delete rawPtr; // dtor // Manage a different object p.reset(new IntHolder{ 456 }); // ctor DebugLog(p->Val); // 456 } // dtor
There’s also a std::shared_ptr
. This is more like a C# managed reference because there can be many copies of it that all point to the same object. A reference count is incremented in the constructor and decremented in the destructor. When the reference count hits zero, delete
is called.
#include <memory> void Foo() { // make_shared makes a shared_ptr // Constructor sets reference count to 1 std::shared_ptr<IntHolder> p{ std::make_shared<IntHolder>(123) }; // ctor DebugLog(p->Val, p.use_count()); // 123, 1 { // Copy the pointer // Constructor increments reference count to 2 std::shared_ptr<IntHolder> p2{ p }; DebugLog(p2->Val, p.use_count(), p2.use_count()); // 123, 2, 2 } // Destructor decrements reference count to 1 // Managed object is still usable DebugLog(p->Val, p.use_count()); // 123, 1 // Destructor decrements reference count to 0 // Calls delete on the managed pointer } // dtor
There are quite a few functions to perform various casts on the managed pointer and create a new shared_ptr
from the result. Here’s one for reinterpet_cast
:
#include <memory> void Foo() { std::shared_ptr<IntHolder> p{ std::make_shared<IntHolder>(123) }; // ctor DebugLog(p->Val, p.use_count()); // 123, 1 // Effectively: make_shared(reinterpret_cast<int*>(p.get())) std::shared_ptr<int> p2{ std::reinterpret_pointer_cast<int>(p) }; // ctor DebugLog(*p2, p.use_count(), p2.use_count()); // 123, 2, 2 } // dtor
A variant of shared_ptr
implementing weak references, in the same sense as the WeakReference
class in C#, exists in the form of weak_ref
:
#include <memory> void Foo() { std::weak_ptr<IntHolder> weak; { // Make a shared_ptr auto shared{ std::make_shared<IntHolder>(123) }; // ctor // Get a weak pointer to the shared pointer weak = shared; // Check how many strong references there are from shared_ptr DebugLog(weak.use_count()); // 1 // Get a strong shared_ptr in order to access the managed object std::shared_ptr<IntHolder> lock = weak.lock(); DebugLog(lock->Val, weak.use_count(), lock.use_count()); // 123, 2, 2 // lock's destructor decrements reference count to 1 // shared's destructor decrements reference count to 0 // shared's destructor calls delete on the managed pointer } // dtor // No more strong references. The managed object has been deleted. DebugLog(weak.use_count()); // 0 // Now we get null when we try to lock it auto lock = weak.lock(); DebugLog(lock.use_count()); // 0 DebugLog((bool)lock); // false DebugLog(lock == nullptr); // true }
The original “smart pointer,” std::auto_ptr
, was removed in C++17 in favor of std::unique_ptr
, std::shared_ptr
, and std::weak_ptr
which were introduced in C++11.
Besides these pointer types, the <memory>
header contains many functions for dealing with memory and pointers. Here are a few of them:
#include <memory> // Get the address of an object even if it has an overloaded & operator struct OverloadsAddressOfOperator { int operator&() { return 123; } }; OverloadsAddressOfOperator oaoo{}; DebugLog(&oaoo); // 123 DebugLog(std::addressof(oaoo)); // Maybe 0000001E2256F794 // Align a pointer for a 4 byte object with 8 byte alignment in a buffer char buf[1024]; void* unaligned = buf; size_t size = 1024; void* aligned = std::align(8, 4, unaligned, size); // Copy to memory assumed to be uninitialized // Similar to memcpy() const char* src = "hello"; std::uninitialized_copy(src, src+6, &buf[0]); DebugLog(buf); // hello // Fill memory assumed to be uninitialized with a value // Similar to memset() std::uninitialized_fill(buf, buf + 6, 3); DebugLog(buf[0], buf[1], buf[2], buf[3]); // 3, 3, 3, 3
Lastly, there’s an allocator
class that encapsulates the default allocation strategy for a type. The class is mostly deprecated, but two of its member functions remain:
void Foo() { // Create an allocator of IntHolder objects std::allocator<IntHolder> alloc; // Allocate room for one IntHolder IntHolder* ih = alloc.allocate(1); // ctor // The IntHolder is now usable ih->Val = 123; // Deallocate the memory // Does not call the IntHolder destructor alloc.deallocate(ih, 1); } // Note: destructor not called
Scoped Allocator
The <scoped_allocator>
header contains just one class, std::scoped_allocator_adaptor
, which is useful as a way of nesting allocators within each other:
// My own allocator type. Always returns the same IntHolder object. struct MyAllocator { // Required in order to be considered an allocator type using value_type = IntHolder; // Object to return every time IntHolder ih{ 0 }; // Required in order to be considered an allocator type MyAllocator() { } // Required in order to be considered an allocator type MyAllocator(const MyAllocator&) { } // Allocates by returning the same object every time // Ignores the count's value and type template <typename T> IntHolder* allocate(T) { DebugLog("MyAllocator::allocate"); return &ih; } }; // Make an allocator that uses MyAdapter and falls back to std::allocator std::scoped_allocator_adaptor<MyAllocator, std::allocator<IntHolder>> aa; // Allocate one IntHolder IntHolder* p = aa.allocate(1); // MyAllocator::allocate DebugLog(p->Val); // 0 // Allocate one more. Returns the same pointer. IntHolder* p2 = aa.allocate(1); // Changing one changes the other p2->Val = 123; DebugLog(p->Val); // 123
Memory Resource
The last header we’ll cover for today was introduced in C++17: <memory_resource>
. This defines what are called “polymorphic resources.” Unlike the allocator classes above, they use runtime polymorphism via virtual
functions to allocate memory. While this may be slower than non-virtual
member function calls like the above allocate
, it allows a single class to be used for all types of allocated memory:
// Everything in <memory_resource> is in the std::pmr namespace using namespace std::pmr; // Derive from the abstract base resource class // We need to implement pure virtual member functions struct MyAllocator : public memory_resource { // Allocate bytes, not a particular type virtual void* do_allocate( std::size_t numBytes, std::size_t alignment) override { return new uint8_t[numBytes]; } // Deallocate bytes virtual void do_deallocate( void* p, std::size_t numBytes, std::size_t alignment) override { delete [] (uint8_t*)p; } // Check if this resource's allocation and deallocation are compatible // with that of another resource virtual bool do_is_equal( const std::pmr::memory_resource& other) const noexcept override { // Compatible if the same type return typeid(other) == typeid(MyAllocator); } }; // Make the allocator MyAllocator alloc{}; // Allocate enough bytes for an IntHolder with its alignment requirements void* mem = alloc.allocate(sizeof(IntHolder), alignof(IntHolder)); // Use "placement new" to construct in the allocated bytes IntHolder* p = new (mem) IntHolder{ 123 }; // ctor DebugLog(p->Val); // 123 // Deallocate the memory, but don't call the destructor alloc.deallocate(mem, sizeof(IntHolder), alignof(IntHolder));
Now that we have a resource class, we can wrap it in a polymorphic_allocator
for compatibility with the non-polymorphic std::allocator
paradigm:
// Make the allocator MyAllocator poly{}; // Wrap it polymorphic_allocator<IntHolder> alloc{ &poly }; // The following is identical to the std::allocator example: // Allocate room for one IntHolder IntHolder* ih = alloc.allocate(1); // ctor // The IntHolder is now usable ih->Val = 123; // Deallocate the memory // Does not call the IntHolder destructor alloc.deallocate(ih, 1);
The Standard Library comes preloaded with a few types of allocators:
// Allocates using new and delete memory_resource* nd = new_delete_resource(); // Performs no allocation at all memory_resource* null = new_delete_resource(); // Allocates sequentially // do_deallocate does nothing // Destructor destroys all memory monotonic_buffer_resource mb{}; // Allocates from pools depending on the size of allocation synchronized_pool_resource sp{}; // Non-thread safe version of synchronized_pool_resource unsynchronized_pool_resource up{};
These are typically combined together to form a chain of memory allocaition by passing an “upstream” allocator in the constructor:
// monotonic_buffer_resource // -> unsynchronized_pool_resource // -> new_delete_resource memory_resource* nd = new_delete_resource(); unsynchronized_pool_resource up{nd}; monotonic_buffer_resource mb{&up};
A global “default resource” defaults to new_delete_resource
but can be customized:
// Get the default resource DebugLog(get_default_resource() == new_delete_resource()); // true // Set the default resource to something else synchronized_pool_resource sp{}; set_default_resource(&sp); DebugLog(get_default_resource() == &sp); // true
Conclusion
We’ve seen quite a range of system integration library facilities today. The <chrono>
header provides us with a lot of time-related classes and functions. We can query various clocks and define durations in a flexible and type-safe way. The <filesystem>
header has a comprehensive suite of functionality for dealing with the file system.
The <memory>
header provides memory-related utility functions as well as the very widely used “smart” pointer types. These go a long way to minimizing one of the top complaints about working with C++: error-prone manual memory cleanup. With types like shared_ptr
, we let destructors take care of freeing memory for us with the peace of mind that they’ll run even when exceptions are thrown so we never spring a memory leak.
Lastly, <scoped_allocator>
and <memory_resource>
provide a great deal of customization to memory allocation. We can choose between virtual
and non-virtual
approaches and even chain together different allocation strategies to best suit our program’s needs.
#1 by Rene Stein on April 28th, 2021 ·
Hi,
this is a really great series of articles.
Did you consider publishing it as a book (ebook) when you will finish the serie?
Thanks for your great work.
#2 by jackson on April 30th, 2021 ·
Hi Rene, I’m glad you’re enjoying the series. I have considered publishing it in ebook format. I’ve never done that before though, so I’ll need to do some research into it to see how, the amount of time it’ll take, etc.