There were several well documented periods of persecution between the time Christianity became a largely Gentile religion, around 100 CE, and Emperor Constantine's Edict of Milan in 313 CE. These were never large scale -- typically, they were the result of an overly zealous tribune or governor or city father trying to get rid of local "undesirables." Even when persecutions were officially under way, there were many places in the Empire where they were pursued with less than patriotic observance to the laws.
Soon after August established the Roman Empire, the official Roman civic religion began to worship the "
genius" of the Emperor, basically the divine protector of the Empire in the form of deified past emperors and, latter on, the living Emperor as well. Making a periodic offering of incense to this divine spirit was required by civil law as proof of loyalty to the Empire, the Emperor and the Roman people. Omitting this civic duty brought your devotion to the state into question. Deliberately refusing to do this was tantamount to treason, as it was seen as willful violation of the good order of the Empire.
For various reasons, Jews were exempted from this civic requirement. Christians claimed the same right, saying that their religion was merely a branch of Judaism. The Jewish leaders disputed this, which resulted in Christians
not being exempt from offering sacrifice to the genius of the Emperor (which is very likely the root of the anti-Semitism found in Christian Scriptures.) When communities were required to turn out for periodic demonstrations of loyalty, many Christians refused to offer the incense, claiming the Jewish exemption. Many got away with it, as most Roman officials did not know the difference between Christianity and Judaism. Many others were arrested and charged with treason. (Some Jews, too, were caught up, but Judaism at this time was very inward and did not recruit to any extent. As a result, Jewish communities tended to stay in larger cities where they were know and did not spread to smaller towns the way Christianity did.) After arrest, local laws and the judgement of local prefects led to a wide variance in punishment. Some Christians were charged with a fine, others endured corporal punishment. In a few time periods, and usually only after proving "recalitrant" in their treason, they were sentenced to death. Some few were crucified, but by the late Empire it was far more likely that they would become fodder in brutal amphitheater spectacles, where crowds enthusiastically watched condemned prisoners be torn to pieces by wild beasts (among many other spectacles.)
By the time of the Edict of Milan and the official end of Christian persecution, only a small handful of Christians had actually been killed. The early martyrologies were written as polemics against the pagans and as morality tales to try and prevent the very common backsliding. It was in the writers' interest to make it seem like they were being persecuted for merely being Christian, which was never actually the case. Rather, Christianity was targeted as a movement that promoted treason. Individual Christians were rarely bothered so long as they were willing to prove their loyalty to the Emperor.