Running complex calculations by harnessing the parallel processing capabilities of CPUs and GPUs can speed up servers while reducing overall power consumption and computing overhead, Scott said. Nvidia is already building graphics cores in current Tegra processors.

"The ARM instruction set is more power efficient than x86. That's why there are people looking to build ARM-based servers. That's why we like ARM in phones, because you get more performance per watt, more performance per square millimeter," Scott said.

It makes sense for Nvidia to push its Tegra chips into the server market, which has higher margins than mobile devices, said Dean McCarron, principal analyst at Mercury Research.

"They have some interesting parallel processing technology that works out for them, and they have ARM, which makes sense for them to pair to go after that class of applications," McCarron said.

Nvidia's target market for server chips could be GPU-dependent systems delivering graphics or mathematical rendering in the cloud, McCarron said. ARM processors are not as proficient as GPUs in performing complex calculations, so Nvidia could end up making trade-offs on its ARM CPU design on power to bring in more performance.