Aides did not immediately say whether the invitation was discussed as the two sat next to each other viewing several race heats Saturday.
"We haven't requested a meeting with North Korea, but if I have any contact with them - in any context - over the next two days, my message will be the same as it was here today: North Korea needs to once and for all abandon its nuclear and ballistic missile ambitions", Pence said.
"The girls have been great, they fit in well with our team", said Marissa Brandt, who was adopted as an infant out of South Korea and whose sister plays on the US team.
Pence also has promised the USA will soon unveil "the toughest and most aggressive round of economic sanctions on North Korea ever".
Pence is viewing the short-track competition, in which eight Americans are participating.
South Korea earlier this year said the agreement failed to meet victims' needs and called for more steps, but Japan insists the agreement should not be changed.
Now that North Korean leader Kim Jong-un has invited President Moon Jae-in to Pyongyang for a summit, sources in the Blue House said they are contemplating how, and in what format, Seoul is going to formally respond.
A senior USA government official says that Moon urged Pence to meet with the North Korean delegation, but Pence left the Olympic venue without doing so.
Correspondents say the invitation puts Mr Moon in a hard position as he campaigned on a promise to engage with the North, but his moves in that direction go contrary to the wishes of his ally the US.
The handwritten invitation was delivered by Mr Kim's influential sister, Kim Yo-jong, at a landmark meeting in the presidential palace in Seoul, before the Winter Olympics.More news: Southwest: Midway flights near normal after de-icing woes
More news: 'Mad development': USA reportedly wiped out scores of Russian fighters in Syria
More news: What to do when Valentine's Day and Lent collide
But North Korea's record on human rights and the growing threat from its nuclear weapons program appeared out of mind.
The South Korean presidential office spokesperson, Kim Eui-kyeom said that Yo-jong came to South Korea as a "special envoy of Kim Jong-un".
Speaking a year ago at his swearing-in ceremony, Moon said he would be willing to travel to Pyongyang "under the right conditions", adding that "for peace on the Korean Peninsula, I will do everything that I can do".
Moon did not immediately accept the offer, calling instead for efforts to "create the right conditions to realise" such a visit and urging Pyongyang to actively seek dialogue with the United States, said the spokesman.
"Everybody's been wonderful, absolutely wonderful", said Mancari of the South Koreans.
"They fit in great with our team, and we're just trying to get them integrated as quickly as possible" said Brandt, a team assistant captain whose Korean name is Yoonjung Park.
Kim Yo Jong arrived in South Korea on Friday for the opening ceremony of the Winter Olympics in the alpine resort town of Pyeongchang, where she had her first face-to-face encounter with Moon.
Moon, for his part, took the opportunity to highlight the visit of North Korean officials to the global competition, referring to the "Olympic Games of peace".
The officials toasted with the Korean liquor "soju" and were served buckwheat and potato pancakes, dried pollock soup, grilled halibut and other dishes. North Korean Jong Su Hyon had one of Korea's three shots in the opening period.