The Festival of Harvest (also called Weeks, Pentecost, or Shavout) is one of the three commanded festivals of Yahweh during which all men are commanded to gather together in a set-apart convocation to worship Yahweh, our Father in Heaven.  For an example of use of the word "convocation" see Leviticus 23:21. The Hebrew word for "convocation" (#4744) is miqra, meaning a meeting or rehearsal that is called.

Exod 23:14 (KJS) Three times thou shalt keep a feast UNTO ME in the year. 15 Thou shalt keep the feast of unleavened bread: (thou shalt eat unleavened bread seven days, as I commanded thee, in the time appointed of the month Abib; for in it thou camest out from Egypt: and none shall appear before me empty:) 16 And the feast of harvest, the firstfruits of thy labours, which thou hast sown in the field: and the feast of ingathering, [which is] in the end of the year, when thou hast gathered in thy labours out of the field. 17 Three times in the year all thy males shall appear before the [master Yahweh].
Festival of Harvest is also called the Festival of Weeks because some translators have chosen to assume that the word "Sabbath" could be replaced with the word "weeks."  The word "sabbath" comes from the word for "cease;" it does not mean "seven" or "weeks."  A sabbath is a day of ceasing from gainful work.  There are regular seventh day Sabbaths, and seven appointed days during the year that are High Day Sabbaths (all "days of ceasing" from gainful work.).

The Festival of Harvest is always on the first day of the week, because it is 50 days from the day after the regular seventh day Sabbath that falls during the Festival of Unleavened Bread.

Levi 23:15 (KJS) And ye shall count unto you from the morrow (day) after the sabbath, from the day that ye brought the sheaf of the wave offering; seven sabbaths shall be complete:
16 Even unto the morrow (day) after the seventh sabbath shall ye number fifty days; and ye shall offer a new meat offering unto [Yahweh].
17 Ye shall bring out of your habitations two wave loaves of two tenth deals: they shall be of fine flour; they shall be baken with leaven; [they are] the firstfruits unto [Yahweh].
The counting is from the firstfruits of barley to the firstfruits of wheat.

The Jewish Calendar counts the days from the Jewish Passover, which they do on Abib 15th as a High Day, instead of Abib 14th as a High Day.  Therefore their Shavout (weeks) always ends up on Sivan 6.  But the scriptures say to count seven complete "Sabbaths" from the day the wavesheaf was offered.  After the seventh Sabbath is completed, 49 days would have passed, and it would be the fiftieth day and the first day of the week.  A difference of opinion exists concerning the meaning of "sabbaths" in Lev 23:15.  Does it mean "weeks" (starting on any day of the week and ending on any day of the week) or does it mean "Sabbaths" (counting from the completion of a seventh day Sabbath through the completion of seven more seventh day Sabbaths).  The Hebrew word in Lev 23:15 is Shab-bawth, "Sabbaths." If "sabbaths" in Lev 23:15 meant "weeks," it would be from the word "shavout" instead of from the word "Shab-bawth."   Indeed, this is what the Jewish people today call it, Shavout, but we see that this is not the word in the Hebrew Scriptures.
Yahweh gave specific dates for Unleavened Bread, Trumpets, Atonement, and Tabernacles; if He wanted the Festival of Harvest to always be on Sivan 6, why wouldn’t it be stated along with these other specific dates?  The relationship keeps the Festival of Harvest on the same day of the week (first day / Sunday), but different dates year to year if you do the Wave Offering during the Festival of Unleavened Bread on the day after the weekly Sabbath. Yahweh did not chose a specific date, but instructed us to count to determine the Festival of Harvest.

1.)  It is the day two loaves of bread were offered as the firstfruits of the wheat harvest.  First, the Festival of Unleavened Bread had the wavesheaf as the firstfruits of barley; this foreshadowed Yahshua himself the firstfruit of man resurrected to eternal form.  Then seven weeks later, Harvest had the wave offering of the firstfruits of the wheat harvest made into two loaves of bread with yeast; a foreshadow of the Hebrews and Gentiles being the firstfruit of a now joined people, the people of Yahweh.
2.)  It was then on this day that the followers of Messiah were endued with power from on high (the Ruach Hakodesh) to help them walk according to  Yahweh’s guidance, direction, and teaching; to thereby overcome the flesh, the world, and sin; and to teach others to also have faith in Yahshua for our atonement.

Roma 8:3 (NIV) For what the law was powerless to do in that it was weakened by the sinful nature, Yahweh did by sending his own Son in the likeness of sinful man to be a sin offering. And so he condemned sin in sinful man, 4 in order that the righteous requirements of the law might be fully met in us, who do not live according to the sinful nature but according to the Spirit.

For those in the upper room after Messiah’s ascension, is was a beginning.  It was the active beginning of the "called out ones," the body of the Messiah; to now work without the personal leadership of the Messiah, but with the supernatural help of the Ruach Ha Kodesh, the set-apart breath, the power from Yahweh.  In the Hebrew and Greek language, these people were called "pure ones" (saints).