His scrambled effort was enough to secure three precious points for the Sky Blues and move the team up to 18th in the SkyBet League One table.
Manager Steven Pressley made two changes from the team that lost to Notts County last week. Aaron Phillips and Reda Johnson came into the side, to replace Jordan Willis and Adam Barton respectively.
The game started cagily, with both sides keen to not let anything slip early. City's 4-4-2 formation was matched by the U's, and chances were scarce.
The first chance of the match came after six minutes. A fantastic route one ball from Ryan Allsop allowed Frank Nouble a chance clean through on goal, but his attempt to flick the ball around goalkeeper Jimmy Walker was scuffed into his arms.
The hosts responded quickly, and put City under pressure as they looked to use the wings to move the ball forward. Phillips was being kept busy and put a great sliding challenge in on Sanchez Watt to stop a U's attack.
City got another chance after 13 minutes. Madine struck a free kick 30 yards from goal, but his effort was high and wide.
It was tough going on a slick pitch but the Sky Blues were making inroads. Jim O'Brien had a half chance after 20 minutes when he attempted a volley from the edge of the area after a Ryan Haynes cross, but his effort went over the bar.
The U's showed glimpses of their attacking threat, and had another effort midway through the first half when George Moncur's angled shot from 20 yards sent the ball fizzing wide.
O'Brien was being productive in midfield, and looked to be the main catalyst for City coming forward. The Sky Blues were also keeping their back line shape well, not conceding a corner until the 33rd minute.
Sanchez Watt forced a smart save from Allsop seven minutes before the break. His free kick had plenty of pace and the City keeper was not going to take any chances and tipped the ball over the top of the bar.
On the stroke of half time City had one of their better chances. Madine struck the ball on the edge of the box after Nouble laid it up, but the ball went looping past the post.
It was an even game that saw few clear cut chances being made by either team, but it was a good foundation for City to build on as they looked to break the deadlock in the second half.
The second half started at a frantic pace, as the hosts looked to attack from the off. A period of heavy U's pressure culminated with a chance for Watt, but his effort went over the bar from inside the penalty area.
After that let off, the Sky Blues looked to fashion an attack of their own. Haynes continued to venture down the left flank, but Sean Clohessy was tracking his man well, and made several vital stops.
On the hour mark City came desperately close to an equaliser. A couple of corners pressurised the U's and on the second attempt Madine got a knock down, but no-one could slide the ball home.
A couple of minutes later saw City go even closer. Nouble powered down the right and cut the ball back to the penalty spot. Madine steamed in and struck it hard and low, but a brilliant block by Marcus Okuonghae stopped a certain goal.
As the game entered the final stages it looked like it was anybodies game and the Sky Blues broke the deadlock with a quarter of an hour to go.
Nouble again caused problems in the U's defence, he played a reverse pass to substitute Danny Swanson. His effort was saved by Walker, but Madine came in to bundle the ball home and send 503 of Sky Blue Army wild.
The U's began to crumble at the back and City had several chances to kill the game off. First Madine nearly got a carbon copy of his first goal, but his effort was stopped on the line by Tom Eastman.
Then Aaron Martin nearly got his first for the club when he flicked a leg at a loose ball following a corner, but he could only scoop his shot over the bar.
City continued to hold defensively and looked for chances to kill the game off and they nearly did it in the final minute. Haynes was released through on goal, but his drive was blocked by Walker.
Four minutes were added on by the fourth official and the U's nearly fashioned a late chance, but more great City defending thwarted them.
The Sky Blues held on, and recorded their first away league win since March, and put them three points clear of the relegation zone.
Attendance: 4433 (503 Away)
Referee: A. Woolmer
CoventryCity: Allsop, Webster, Johnson, Fleck, O'Brien © (Swanson 71'), Phillips, Nouble (Barton 84'), Haynes, Martin, Madine, Finch
Substitutes: Charles-Cook (GK), Thomas, McQuoid, Maddison, Miller
ColchesterUnited: Walker, Okuonghae ©, Watt (Healey 64'), Gilbey, Sears, Eastman, Clohessy, Massy (Roofe 69'), Moncur, Fox (Szmodics 69'), Kpekawa
Subsitutes: Lewington (GK), Eastmond, O'Donoghue, Hewitt