Levi 23:26 Yahweh said to Moses,

27 "The tenth day of this seventh month is the Day of Atonement.”

Levi 23:32 (NIV) It is a sabbath of rest for you, and you must deny yourselves. From the evening of the ninth day of the month until the following evening you are to observe your sabbath."

So the beginning of the 10th day is scripturally described as from the evening of the 9th.  If evening were at the beginning of a day, Levi 23:32 would force us to include all of the 9th day as part of the 10th day; therefore, evening must be a period at the end of a day.  Hence, Scripture shows us that the evening of a day is always a time at the end of the day.

This seems to force a definition of “evening” as a period of the day before sunset, since the next day begins just after sunset. This also seems to force a definition of that beginning part of a day just after sunset as “morning” (even though is will be getting dark and then be dark until the sunrise).

Exodus 12:6 says to kill the Passover lamb on the 14th day of the 1st month in the evening.  Evening is a period of the day before sunset.  See above.

(Ref. 1): "In the evening" in the Hebrew would be said: "bain hahaarbayim" (literally translated "between the evenings”). This phrase suggests the time, a point between the sun's declining in the west and its setting, (approximately 3:00 p.m.). Within the temple the day was divided into quarters. The quarter between 12:00 noon and 3:00 p.m. was called the minor evening oblation, while that between 3:00 p.m. and 6:00 p.m. was called the major evening oblation.  Therefore, "between the evenings" means between those two periods or 3:00 p.m. The hours were counted from daylight (approximately 6:00 a.m.) till sundown (approximately 6:00 p.m.).  The ninth hour when Yahshua was slain would therefore be 3 p.m., the same time the lamb had been slain in Egypt.

(1)   "Rosh HaShanah and the Messianic Kingdom to Come" Hatikva Ministries 1989, Joseph Good.