The world of video games keeps getting bigger and more populous, and not just with other players. Open-world sandbox games thrive because hundreds of pre-programmed non-player characters (NPCs) make a fake city look like a real one. And they’re only getting more lifelike.

Assassin’s Creed: Syndicate was released this month, a game where the player’s purpose is to covertly move around in public while murdering specific targets. If you’re doing a good job, none of the NPCs will treat you differently from the rest of the “crowd.”

This is a fairly new dynamic in gaming: Most of the early-decade games were filled with NPCs that helped, guided, or hurt the player on their quest. You’d either have teammates in a sports game, enemies in a shooter, or creatures to kill in a role-playing game (RPG).

When you look at first-person shooting games like Call of Duty or Halo, the main purpose of NPCs is to interact with you. To put it another way, their primary objective is to find you, shoot at you, and only stop when you are dead (or they are dead). For years, game designers have been focused on making these interactions smarter: does the NPC take cover when you begin shooting back? Do they coordinate with others? Do they sneak up behind you? How they respond to the world around them has been of growing importance as shooting gamers desire “smarter” enemies.

This is a binary relationship between the player and enemies. It extends, likewise, to the cooperating “allied” NPCs whose primary purpose is either to help you or to let you help them.

But there is a third and less talked about category—something in between: the NPC whose primary purpose is to ignore you.

In a game like Grand Theft Auto (GTA) or Assassin’s Creed, pedestrian NPCs have a sort of pre-programmed set of responses to the environment. If you’re not interacting with them, they’re meant to reproduce the kind of crowd flow and individual actions that you’d see on a sidewalk in a major city, moving from point A to point B without walking into walls, other NPCs, or you.

Usually, these characters are meant to look normal as you’re speeding around city streets, and all flee in unison when you begin firing guns at whatever takes your whim. So when you don’t cause any disturbances, and when you keep them in sight for extended periods of time, what exactly do they do?

Pedestrians in GTA don’t have relationships or identities. They don’t have jobs (except the police and a few others you’ll interact with directly), hobbies, or homes.

But maybe they need to.

The new challenge though is that, as open world games become a place for players to wander aimlessly, pedestrians need to have goals. An Imgur user recently spent a few hours following around an NPC dog, just to see what it would do.

In case you don’t have five minutes to scroll through a photo essay: The dog eventually took a wrong step, fell, and died. Our player in question had developed a bit of an attachment to the dog, without anyone at Rockstar (the game’s studio) intending for him to do so. Rockstar’s job is to fill streets and alleys and forests with the kinds of people you’d expect to see as you drive by. Had the player driven away, the dog would have eventually been out of range, de-spawned, and reset.

But maybe someday, with big enough processors, these people (and animals) won’t de-spawn. And at some point we’ll have a game where you can follow a dog, or a random dude, all the way back to his office and watch him sit down at his desk.

To be clear, we’re years from every little no-name polygonal person in a game like Grand Theft Auto having a personality and thinking. Artificial intelligence may come, but implementing it within a virtual world on a scale like that would be a stunning feat, even for the likes of a major studio like Rockstar (who did not respond to a request for comment).

But mimicking those responses without letting them happen in real-time seems to be getting easier, as NPCs do things like wait to cross a street until the light turns, or stop to pick up a dropped coin on the sidewalk.

It’s still easy to fire a rocket launcher into a crowd of functional-yet-dimwitted NPCs, because that’s really their purpose: to be disturbed, harmed, or forced to react.

The thing about NPCs is that they’re sort of aimless, anonymous creations, and we can recognize that. But as technological advances give them a personality and a history and a purpose, it’s going to change the dynamic of a game like GTA.

For now, enjoy the more realistic lemmings populating your game worlds. Because there’s reason to expect that sometime in the future they might end up having more purpose in an hour, in your game, than you do.