Regular time saw Germany possess the ball more, orchestrating pass-a-ton attacks while Italy relied on fast breaks.
Mesut Ozil, the brain of the German national team, took advantage of a quick ground cross from the left-hand side to tap in the opener in the 65th minute.
Italy equalized in the 78th minute after Germany's Jerome Boateng handled the ball in penalty area and Leonardo Bonucci converted the kick from the white spot.
With regular time ending 1-1, the game headed to a 30-minute extra time in two halves. No goals from either team meant the semifinal ticket would be decided by penalties.
The shootout was highlighted by misses from big names including Simone Zaza, Graziano Pelle, Matteo Darmian and Bonucci on Italy.
German miscues came from Thomas Muller, Ozil and Bastian Schweinsteiger, meaning that both goal scorers during regular time missed their end-game penalties, with the two-out-of-five runs failing to impress.
On the ninth try against Italy in major tournaments, Germany registered their first win after their goalkeeper Manuel Neuer made a brilliant save and Germany's ninth penalty taken by Jonas Hector went past Gianluigi Buffon to seal the deal.
Germany is the third qualifier for the semifinal round after Portugal and Wales.
