Getting a piece of writing published online is easy -- all you need is an Internet connection and there's an endless amount of places to put your words. Getting a piece of writing published in a literary magazine or journal, however, is quite the opposite. So... why go through all the trouble?
Find a publication that matches your type of writing.
This is excellent advice from Poets & Writers, which cautions that writers should research magazines and journals before submitting any work to them. See what kind of tone, length, voice, and topics are in the publication currently -- do these seem to be a match for your style of writing? Could you write something to match these things? Would you want to read this publication regularly? Ask yourself these questions before submitting.
Craft something that you absolutely love.
Though you want to follow the first step here and research where you're submitting, you don't want to craft something that you don't absolutely love. Because magazines and journals are often read by agents and publishers, you want the work to be a good representation of what you're about -- something you're very proud of. Seems like a no brainer, but it's a top piece of advice from The Write Practice.
Read the rules and guidelines carefully.
Not following the rules and guidelines for submissions is a quick way to get tossed out of the running. Themes, word limits, genres they don't want, deadlines -- some even specify exactly how they want the submission arranged and what extra info they want from you. You want to be noticed -- but not for ignoring the rules that magazines clearly post.
Make connections and self-promote -- without being over the top.
Always good for a writer to network, but one important reason is because you might get your foot in the door with someone who works at a literary magazine or journal. You don't, however, want to be too persistent with these leads, as Cosmopsis explains -- you don't want to nag or bother someone too much, so create personal relationships with people in the industry and then talk business.
Don't ignore non-paying gigs.
Obviously getting published and making money is ideal, but don't ignore magazines and journals that don't pay. It could be a good way to test the waters and grow your portfolio.
Read, re-read, and re-read again before sending something in.
Remember -- you don't get a chance to defend your piece in person, so edit it carefully for grammatical mistakes and typos. Also, read it aloud and make sure that it makes sense and is very clear for the reader. Work out awkward sentence structure and holes in the story before you send the finished product -- if something doesn't make sense to you, it probably won't make sense to an editor at a magazine.
Don't ignore rejections.
If you get a rejection letter, don't toss it aside -- instead, follow The Review Review's tip and take it as a good sign. If an editor actually takes the time to offer feedback and comment on your work, that person probably liked it -- start a conversation and ask if you can send a revision, send something else, or get clarity on the feedback. The editor might start a dialogue, which is a great way to start a relationship with the magazine and get really helpful feedback on your work for next time.