The std::unordered_set class template represents a chained hash table--that is, a fixed-size array of "buckets," each bucket containing a singly linked list of data elements. As new data elements are added to the container, each element is placed in the linked list associated with the "hash" of the element's value. This is almost exactly the same as Java's HashSet. In memory it looks like this:
The literature on hash tables is extensive, and std::unordered_set does not represent even remotely the state of the art; but because it eliminates a certain amount of pointer-chasing, it tends to perform better than the tree-based std::set.
To eliminate the rest of the pointers, you'd have to replace the linked lists with a technique called "open addressing," which is...