Linux wasn't my first unix-like OS, that goes to FreeBSD.
With linux I started from debian, like 12 years ago, then geting fed up dpkg bullshit went to slackware and did some lfs runs. After that I was ready for gentoo, then switched to funtoo for having a bit nicer repos. But then gentoowiki was lost, gentoo and it's forks withered down and on slow decline now. But in meantime a very similar in some concepts distro arose and that was arch.
It too gained a lot of popularity by having most detailed wiki of them all, but most importantly: it has package build system that is very friendly, just like with ebuilds you can easily define building a package in a single simple shellscript aided by library functions, allowing for easy software customizations. No system-wide use-flags however, universal optimization is a bit trickier than in gentoo, but not really required. So Last 6 years or so i am solely using arch.
Then, there is AUR, a user repository of unofficial pkgbuilds, also integratable with package manager.
And then, there is binary builds for every official package, so you don't have to wait half a day while your chromium is compiling.
So, instead of a distro to settle I suggest a journey to find one that suits you. Walking through LFS is a good quest on this path, yielding a considerable knowledge, give it a try:
http://www.linuxfromscratch.org/lfs/view/stable/