Representatives of British in Europe, an umbrella organisation representing the 1.2 million British people living on the continent, said the loss of rights was "already real" for many people living abroad.
UK nationals have free movement in the EU until the end of the transition period in December this year, giving them the right to live work and study abroad.
But EU employers are anticipating the end of those right, which will make it harder for Britons to do some jobs that involve cross border travel.
"We are seeing people refused interviews for jobs because those jobs require the freedom to travel across the EU," Kalba Meadows, a steering group member of British in Europe who lives in France, told the EU Future Relationship Select Committee.
"We're not even at the end of transition yet and there are already real live instances of people's lives and livelihoods being affected – and that will only increase. So many jobs rely on free movement because of the single market."
Ms Meadows added: "We have a very large number of people whose livelihood is based on working in different countries - people with small businesses, people who are employed, and an awful lot of them face losing our livelihoods. It's not just about losing your rights on paper, it's something that affects real lives."
While the Brexit withdrawal agreement protects some of the rights of British citizens abroad and EU citizens in the UK, there are holes in it, notably with regard to jobs not based in a single member state. The integrated nature of EU economies means that occasional travel to other countries is common, particularly in professional-level occupations.
There are also gaps in the extent to which Britons' qualifications will be mutually recognised in the EU after Brexit.
Michael Harris, a steering group member of British in Europe who lives in Spain told MPs: "There's a stereotype of a Briton who lives in Spain - there are a lot of retired people in Spain who won't be affected by the need to go and work or provide cross-border services, but around 60 per cent of Britons in Spain are working age or below. Young Britons in the EU are the people who are going to be most affected by this - to go and study and work."
Jane Golding, the chair of of the organisation, told the same committee on Tuesday that the suggestion that rights for EU nationals after Brexit were the same as before was misleading.
She said the rights were in fact "broadly the same only in the host country where we are living now".
"So that means that we will keep most of our rights that we currently have in the country where we live now but we will not have any EU-wide rights of free movement, for example, or EU-wide recognition of our qualifications. There are no rights in the withdrawal agreement on cross-border working," she said.
Britain and the EU are currently negotiating a future relationship, which both sides have always said will cover immigration issues. But political pressure in the UK has led to the government ruling out a continuation of free movement.
Neither side's draft agreement includes provisions for free movement rights and the UK's draft immigration bill contains no provisions for such policies.
Ms Golding told the committee: "We haven't had any indication from either side that this is a topic that's being discussed in any detail in the future negotiations."
