(The web is the most common vector for malware. You need to know how they are protecting their computers and servers from malware infection.)

What about externally facing web--what have you done to secure the application or harden the interface? Do you have regular application penetration tests to assess the real-world security of your internet-facing applications? Can I see the report?

(This speaks to their maturity of Web Application Security. Ask if you can see a copy of the report.)

How do you protect the data and know where it is going? Are you using DLP technology? What kind of encryption scheme do you use to protect my data? How is my data segregated from other customers?

(You want to make sure data is only going to approved users. Make sure the data is encrypted at every stage in the transaction, not just via SSL while in transit.)