There’s no limits programmed into EE but to reflect Boyink’s input there are limits related to how SQL and servers themselves work. Since EE is designed to be very flexible it’s SQL queries are not always going to be the most optimal for certain kinds of tasks (partly because to obtain that flexibility the database structure has been designed a certain way) and I’ve seen it break down when attempting to retrieve large amounts of entries or when retrieving a few entries from a large weblog with many categories.
Usually you can code around these sorts of issues using caching and template changes. But there are limits to performance in SQL where just getting a faster server doesn’t really help anymore.
These are the same sorts of limits you see Facebook and Twitter dealing with all of the time.
In a case where you have something incredibly specialized like a payroll system I agree with Boyink I’m not sure EE is what you want. Or that using built in functionality within EE is what you want. A custom module with it’s own database structure and tables would probably make a lot more sense if for some reason you really wanted this to be in EE. That way you can design the tables around the sorts of tasks that a payroll system would require and be more sure that it’ll hold up as the database grows.
Jamie