The whole basis of the "right to immigrate" argument starts with a kind of out-of-context, rationalistic, assertion of individual rights from which proponents proceed to deduce conclusions. The proper way to deal with this question is inductively, i.e., start with the facts that give rise to individual rights in the first place so we can see how to apply the concept in practical situations.
While I take as a given that individuals possess natural rights, by virtue of being human, the fact is that individuals must take specific actions in order to protect those rights in a social context. Rights are not respected out of thin air. There is not an ethereal entity that emerges from the universe to protect our rights. There needs to be an actual means or vehicle through which we attempt to protect, as best as possible, the natural rights to which we are entitled.
When a group of property owners and individuals agree to form a government to protect each other's rights and provide for the common defense, they create a sovereign "nation" by virtue of that agreement. That government, like a home owner's association, is granted certain enumerated powers, and that particular government, in effect, is created by and for those particular people, its citizens. That particular government can take many different forms but, in essence, it is the framework by which those people choose to protect their natural rights.
Those people and their government are a sovereign nation. They elect and grant power to the government to represent them in dealing with foreign nations or foreign nationals. The reason they do this is to protect themselves from threats and deal with affairs of state that affect the group collectively. The threats and issues stemming from foreign relations can take different forms and a foreign policy must be defined, and laws implemented, to fairly and objectively deal with foreigners while respecting the rights of its own citizens.
Note the flow here. It starts with property owners and citizens seeking to protect their rights by forming a government within their lands. The people, through their government, then look out at the world and say "how do we deal with the rest of the world beneficially, while protecting the rights of our citizenry?" It does not start with the claim that "everyone has rights" and proceed to the conclusion that "anyone at anytime can come into any sovereign nation irrespective of the laws and policies of that nation because rights."
A sovereign nation is comprised of the private property owners and individuals that elected and fund that government within that border. The relationship of the foreign national to a sovereign nation is analogous to the relationship of random citizen to a private property owner. In the case of a private property owner, he can determine who comes onto his property and with whom he may trade or cooperate. But, just because someone rings his doorbell, it does not mean he must grant him entry.
In similar fashion, the government is elected to act as the gatekeeper at the nation's private property door. However, complications arise, because the government represents a group (the country), not a single individual, and therefore must represent the interests and protect the rights of all the citizens of the sovereign nation in determining who can enter. Since the government represents everyone, its actions must be based on objective policy and law.
For example, if one private property owner wishes to trade with foreigners or wishes to bring foreigners into the country, can he do so without the implicit or explicit consent of the rest of the country? This is exactly why foreign policy and immigration policy is necessary. It is designed to deal with exactly these situations, and it is why it is a necessary function of the government in its capacity to provide for a common defense.
In a detail prior post, I covered, in some detail, the types of issues and the types of threats that would be of concern under such a policy. There I define "threat" in the foreign policy context, and stress the necessity to identify such threats contextually. For example, should the government prevent a private company from selling military technology or weapons to North Korea or Iran (maybe in the future, these countries will be allies and that is fine)? Can a private property owner bring in individuals who carry a deadly virus? Can a private property owner bring in and house thousands of communists who wish to overthrow the government? Could a private property owner who owns ocean frontage allow an enemy nation to dock its nuclear submarine fleet and board its military personnel? Can private property owners transact with nations under embargo? Can a private property owner bring in ten million destitute Muslim migrants because he feels sorry for them?
Consider other questions related to the rights of immigrants. If it's true that we must protect the rights of foreign nationals outside the country (before they are granted residency or citizenship), should we grant those same rights to foreign nationals before launching an attack on an enemy country? Would we interview each foreign national and grant them a jury trial and the various constitutional protections specified by the Bill of Rights before the attack? If you say, "war is different" then you are supporting my argument - foreign policy and the relationship of our nation to foreign nations is a different legal context. The policies by which our country deals with foreign nationals is and should be reasonable and a matter of law (not fiat), but the concept of natural rights does not impose an obligation or duty on the citizens of a nation to extend its own government's jurisdiction to anyone who asks for any reason.
In the linked post, I make the case that the burden of proof falls on the immigrant or visitor, or his sponsors, to prove they are not a threat or will not impose obligations on others. If that immigration criteria (developed by the elected representatives of the people) is met, i.e., they are found to not represent a threat under the policy, then it is reasonable for someone to immigrate or visit. After all, to deny a citizen within the country who seeks to peacefully deal with a non-threatening foreign national, could be considered a violation of that citizen's rights (he would rightly ask, "why would you all stop me from trading or marrying this nice person who I can prove is a great person and no threat to you?"). But, note, any right's violation is considered from the perspective of the citizen, not from the perspective of any right to immigrate.
Of course, if this policy is appropriate in a more free market context, how much more important is immigration policy today, where we do have a massive welfare state, where democratic elections can be dramatically affected by immigrant voters who skew left, and where myriad threats from overseas endanger our country? Again, the facts and context drive the state's policy which exists to protect the rights of its own people.