Three boys, Sean Devine, Jimmy Markum, and Dave Boyle were playing hockey in the street. While playing, the boys find a section of sidewalk concrete that is still drying. Jimmy impulsively writes his name in the cement and Sean follows. Dave begins to write his name in the cement, but a car pulls up and a man who pretends to be a plainclothes police officer gets out, scolds the three boys, and tells Dave to get in the car. As the car moves away, Dave looks out the back window to see Jimmy and Sean staring back at him. Hearing that Dave was taken away by a police officer, the parents of Jimmy and Sean agree that something is wrong and begin to look for him.
The film then cuts to Dave in the basement of the two pedophiles who took him and then to Dave running away from the house through a forest.
Decades later, they are all still living in Boston. Jimmy (Sean Penn) is an ex-con running a neighborhood store, while Dave (Tim Robbins) is a blue-collar worker, still haunted by his abduction. The two men are still neighbors and related by marriage. Jimmy's 19 year old daughter Katie (Emmy Rossum) is secretly dating Brendan Harris (Thomas Guiry), a boy Jimmy despises. She and Brendan are planning on eloping to marry in Las Vegas.
Katie goes out for the night with her girl friends and is seen by Dave at a local bar. That night, Katie is murdered, and Dave comes home with an injured hand and blood on his clothes, which his wife helps him clean up. Dave claims that he fought off a mugger and possibly killed him. Sean (Kevin Bacon), who is now a detective with the Massachusetts State Police, investigates Katie's murder with his partner (Laurence Fishburne). In a subplot, Sean's wife Lauren has left him, and subsequently telephones him without speaking. She is pregnant while she makes this call, but won't even tell Sean the baby's sex.
Over the course of the film, Sean and his partner track down leads while Jimmy uses his neighborhood connections to conduct his own investigation. Sean discovers that the gun used to kill Katie was used in a liquor store robbery during the 1980s by "Just Ray" Harris, the father of Brendan Harris, causing him to suspect Brendan. Ray Harris has been missing for some time, but Sean believes that his gun was still in the house. Brendan claims that Ray has been sending $500 a month since he disappeared. Sean also learns that Jimmy is listed as a known criminal associate of Ray Harris. Sean's partner suspects Dave, as he was one of the last people to see Katie alive and has a wounded hand (Dave tells them that he hurt it in a garbage disposal).
Dave continues to act strangely and his wife eventually tells Jimmy about Dave's strange behavior and the bloody clothing. She tells Jimmy that she thinks Dave killed Katie.
The climax of the film occurs when Jimmy and his friends get Dave drunk. When Dave leaves the bar to vomit next to the Mystic River, the men follow him out. Jimmy tells Dave that he shot "Just Ray" Harris at that same location for ratting him out and sending him to jail. This caused Jimmy to be absent while his first wife was battling cancer and ultimately dying while he was in prison. Jimmy tells Dave that he will let him live if he confesses to killing his daughter; if he does not he will kill him right then and there. Dave repeatedly tells Jimmy that he did kill someone but it was not Katie: he killed a pedophile, after finding him with a child prostitute in a car. Dave is so nervous he vomits once again. When Dave finally admits to killing Katie in an attempt to escape with his life, Jimmy stabs him in the stomach and shoots him in the head. They dispose of his body in the adjacent Mystic River.
While Dave's murder is occurring, Brendan (having found out about his father's gun from Sean during questioning) confronts his younger brother and his brother's friend about Katie's murder. He savagely beats the two boys, but is almost shot by one of them when Sean arrives just in time to stop it.
The next morning, Sean tells Jimmy that the police have Katie's murderers who have confessed. She was killed by Brendan's brother and his friend in a violent prank gone wrong. Sean asks Jimmy if he has seen Dave, because he is wanted for questioning in another case, the murder of a known pedophile. A distraught Jimmy thanks Sean for finding his daughter's killers, but says "if only you had been a little faster". Sean asks Jimmy if he is going to send Celeste Boyle $500 a month too, as he had been doing for the widow of "Just Ray" Harris. This is why Brendan believed his father was sending his mother the money every month, even though it was clear he was dead. Jimmy's wife comforts him over Dave's murder, telling him that he did what he had to do because he loves his daughter. Later at a parade Dave's wife Celeste frantically tries to get the attention of her despondent son Michael (Cayden Boyd). Sean spots Jimmy in the crowd and makes a gun with his hand, 'shooting' it at Jimmy. Jimmy shrugs and puts on his sunglasses.