The children of Israel (Jacob) remained in Egypt even after the famine was over.  They had many children, and generations later became very numerous.  The new Pharaoh did not remember Joseph and what he had done to save Egypt from the famine.  Instead he resented that there were so many foreigners in his country, so he had all the descendents of Israel (called Israelites) enslaved.  He put taskmasters over them who made them build for Pharaoh.