Wales was level 16-16 with five minutes to go in the semifinal against South Africa but lost on Handre Pollard's late penalty goal. The Welsh are now 0-3 in World Cup semifinals, and will be playing for bronze against the three-time champion All Blacks.
"We punch massively above our weight in terms of the playing numbers in Wales, so I'm really proud of these guys, they've given us 100%," Gatland said. "We're disappointed, but I think we can hold our heads high and leave Japan with a lot of respect."
Gatland has guided Wales to a Six Nations Grand Slam, and led the British and Irish Lions on successful tours. New Zealand is the only top-tier team he hasn't beaten as coach of Wales and he says "It'll be monumental."
Handre Pollard landed a penalty with four minutes to go to lift South Africa to a 19-16 win over Wales and into a Rugby World Cup final against England.
After a tryless first 56 minutes, each team crossed within the space of eight minutes to make it 16-all with 14 minutes left in regulation.
It was 9-9 when center Damian de Allende bustled his way over for South Africa after beating two defenders wide on the left. Pollard, who made two probing runs in the leadup, converted to make it 16-9.
Wales turned down a chance at a long-range opportunity at goal to chase a try, and got a lineout close to the left touchline. After 18 phases failed to breach the South Africa defense, Wales was awarded a penalty and took the scrum. South Africa got the shove, but Wales cleared the ball to the left wing and Josh Adams dived over to make it 16-14.
Leigh Halfpenny calmly slotted the conversion from three meters in from touch to level at 16-16.
Both teams had attempts at dropped goals before Pollard sealed it with a penalty goal in the 76th.
Two-time champion South Africa will meet England in a rematch of the 2007 decider.
South Africa leads Wales 9-6 after an attritional first half of a Rugby World Cup semifinal that was contested with high kicks and a constant tussle for territory between set pieces.
Springboks flyhalf Handre Pollard kicked penalty goals in the 15th, 20th and 35th minutes and Dan Biggar responded for Wales with a shot from near the left touchline in the 18th and another goal a minute before the break.
Both teams play a direct style, with two-time champion South Africa priding itself on the set-piece. Both teams struggled on a damp surface for cohesive attack when the ball did go through the hands in a grinding opening 40 minutes.
The injury-plagued Welsh squad had more damage to contend with as prop Tomas Francis left the field with a shoulder injury and veteran winger George North limped off with a right hamstring problem just before the interval.
The winner will return next Saturday for a final against England. The loser will play New Zealand for bronze.
South Africa has kicked off in the semifinal against Wales to see which team advances to a Rugby World Cup final against England.
Players from both teams sang their anthems proudly and loudly in a packed International Stadium at Yokohama.
Wales has never won a semifinal at the World Cup. South Africa has won two titles.
England has called up Ben Spencer to its World Cup squad as injury cover for fellow scrumhalf Willi Heinz.
Heinz hurt his hamstring in the semifinal win over New Zealand on Saturday. England says he will remain with the squad until the end of the tournament.
Wales and South Africa meet in Sunday's semifinal at the Rugby World Cup for the right to face England in the championship match.
If the Welsh get through to their first final, it will be the first time that two northern hemisphere teams face off for the title.
The Springboks have reached two finals before — in 1995 and 2007 — and went on to win the cup on both occasions. In '07, they beat England in Paris.
It is not expected to be a running rugby classic, with both teams preferring to play a low-risk style and rely on forward power. It is an approach that has served both teams well — they have lost just one competitive game between them this calendar year.
England reached the final by beating New Zealand 19-7 on Saturday, ending the All Blacks' ambitions of winning the World Cup for an unprecedented third straight time.
More AP Rugby World Cup: https://www.apnews.com/RugbyWorldCup and https://twitter.com/AP_Sports