This trial is studying a combination of pemigatinib and durvalumab in patients with a specific type of liver cancer called intrahepatic cholangiocarcinoma. Patients will receive the medications in cycles, with pemigatinib taken orally daily for 2 weeks followed by 1 week off, and durvalumab given intravenously every 3 weeks. The patients will have regular check-ups and disease assessments every 9 weeks. Treatment will continue until the cancer progresses, there are side effects, or the patient or doctor decide to stop. Patients can receive a maximum of 24 months of treatment.