I am assuming you are using Microsoft Windows XP.
Just you need to click of properties of the NIC, called Local Area Network in Network Connections.(comes by right clicking Network Places in Start Menu),

Choose Internet protocol, and set up static IP, subnet mask, gateway address, DNS for each of the computers. Look for gateway and DNS and subnet mask in the existing computer configured for Internet OR ask from ISP, and keep a common subnet for all the PCs while IPs should be different, say make it like 192.168.1.2 and further (assuming router IP/gateway IP is 192.168.1.1)