I am fairly new to Dungeons & Dragons. While making my character (Half-Elf Rogue) I thought that it would be cool if my PC owned a pet to tie into her background, as my background as an Urchin gives me a pet mouse. However, is it inconvenient/more complicated for the DM in any way if I do this?

You should be fine.

If your background states that you get a pet mouse, you get a pet mouse. This doesn't mean you get a familiar, though. Your pet will likely not be used in combat, and doesn't have the features of a familiar, but if you're a rather cruel person, you can throw it at traps.

It shouldn't be too inconvenient for the DM since pets don't require much micromanagement on their part.

Speaking as someone playing a Druid accompanied by (currently) five pet badgers, you're fine.

D&D (especially 5e) is set up with a sliding scale of granularity. If your campaign is detail oriented, your DM could make you account for the grain you're feeding your mouse with, or where the mouse is at all times (to be sure that it isn't crushed when you're slammed with a club) or various things like that. But many groups don't find that level of detail interesting, and it's perfectly fine to just handwave the mouse's existence - assume that your rations include mousefood, and that you've found a safe place for the mouse where it can't be harmed during combat. With a setup like that, the mouse isn't adding any additional overhead to your DM's workload, because you're just conveniently assuming that it's fine unless the DM wants to use it as a plot point.

If you can carry 25 longbows without it affecting your combat ability, you can carry a single mouse. And if necessary, you can use magic to further handwave your difficulties away. My druid recently acquired a Backpack of Badger Carrying (which is identical in every way to my previous backpack except that it has an infinite weightless storage space for badgers) because as I gained badgers it was becoming increasingly implausible that we were taking them with us as we scaled cliffs and rode around on dinosaurs.

This is exactly the sort of thing that backgrounds are for.

Part of character creation in 5e involves choosing a "background" for your character. This is partially to encourage players who more mechanically inclined to think about the story aspects of their character, but it's also there to provide a framework for exactly how much mechanical advantage a player should be able to get from their past history. A background comes with 2 skills, 2 tool proficiencies or languages, and 1 "feature".

Features can be anything from "You have contacts in the criminal underworld" to "you're really good at using maps and foraging for food". They're meant to be small but flavorful advantages that are explained by your background.

My Druid's background was "Badger Hermit", with the feature that I had trained my badgers to respond to many different commands. I haven't used it much yet, but it provides a framework for how much I can get my badgers to do on their own if the need arises. If you want your mouse to be more relevant than a rock with sentimental value (which happens to be alive), then talk with your DM about working your relationship with your mouse into your background feature.

It depends on how you play it.

All DMs can eventually get overwhelmed at the table when PCs have lots of NPC followers, or animal companions, or familiars, or pets, or stuff... A pet mouse is low on the objective complexity scale but you can still make it a problem, or not, through play.

The rule for all of these is “is it being a distraction, slowing down the game, and taking spotlight time?” If it’s used in combat, and every RP scene, and so on, then it will bother the DM and players.

In one game I ran, a PC similarly had a mouse. It ended up starting a massive bar fight - they were in a bar with a beloved mascot cat, he got the mouse out to “play with it” (deliberate complication), the cat attacked, another PC “hit the cat with my sheathed rapier” and critted and laid it out, starting a general melee.

That was fun and a memorable gaming story (one player didn’t take it entirely in good humor, but nevertheless). But if he had done that all the time - “Now as we’re meeting with the King, Chirpy climbs into his wine goblet!” “Now as we fight the IRC’s, Chripy scampers around through the melee!” then everyone would have been more like “hey this isn’t the story of your damn mouse, knock it off.”

So have a pet, pop it out around the campfire or in the inn or other quiet RP opportunities, but don’t have it interjected into things all the time.

If it makes you feel any better, check this out: Too many pets...

Your one non-combat pet mouse is not a concern. As a DM, I'd actually love it. It adds depth to your character and is basically a bonus vector for RP opportunities and even plot.

For even more on pets, check out Jester on critical Role. She has like three pets, none of which are familiars or animal companions on her sheet, just animal npcs, and all are more obtrusive than a mouse.