Sgt. Michael Joyce said the teacher told the man to leave and called police immediately at about 12:30 p.m.
The white man, in his late 20s or early 30s, drove away in a green Honda Element, Joyce said. The teacher did not get his license plate, he said.
Police canvassed the area, but did not find him. Joyce said they don't believe the man was related to any of the students.
He said there have been no other incidents reported recently, though similar instances have been reported in years past at other school districts around the South Fork.
East Hampton School District notified parents on Friday afternoon, according to messages obtained by Patch.
District Superintendent Rich Burns sent out an email giving parents a description of the vehicle and the man, who he said was between the ages of 28- 35, had black hair, olive complexion, and brown eyes, and an average build."
"It is a car with lots of debris in it, and what looks like coasters for moving furniture," he wrote.
Burns said East Hampton police were made aware of the situation, however Village Police Captain Mike Tracey said that East Hampton dispatchers were not notified because no alert was issued by Southampton Town police.
Tracey said police heard an automated message from the East Hampton School District about the incident, but that information did not come from police.
"Please remind your students to be safe, and not to speak to adults in cars that they do not know," Burns said in an email to parents.Joyce said police will offer extra patrols of school districts. Anyone who sees anything suspicious is asked to call police at 631-728-5000.