I think your requirement isn't something that you can do with a validation rule and a trigger is going to be your best option. This solution is assuming that the trigger needs to only fire when the opportunity is updated. It relies on the opportunity existing since a new opportunity won't have any line items associated to it so users should never be able to set the config required flag on insert. That could be done with a validation rule easily.

To check when an opportunity already exists I put together the following code. I did this purely from my head and have not tested it so I can't say that it works but it should give you an idea of what you need to to do accomplish this task. 

trigger ConfigProductValidation Opportunity (before update) {
	List<Opportunity> listOpps = new List<Opportunity>();
	Set<Id> setOppIds = new Set<Id>();
	
	//Filter out Opportunities that do not require config
	for (Opportunity opp : trigger.new) {
		if(opp.Config_Required__c == true) {
			listOpps.add(opp);
			
			//add the id to a set to use in the where clause of the future query
			setOppIds.add(opp.Id);
		}
	}
	
	//Requery Opps with Line Items into a Map so that the error can be returned to the trigger
	//instance of the opportunity.
	Map<Id, Opportunity> mapOpps = new Map<id, Opportunity>([SELECT Id, (SELECT Id, Name FROM OpportunityLineItems WHERE PricebookEntry.Product.ProductType = 'Configuration') FROM Opportunity WHERE Id IN: setOppIds]);
	

	//Loop through listOpps
	for (Opportunity opp : listOpps) {
		Opportunity mapOpp = mapOpps.get(opp.Id);
		
		//Checks to make sure that the opp was in the map list and makes an assumption that
		// if the related opportunity line items is null then there are no products that have a product type
		// of Configuration.
		If (mapOpp != null and mapOpp.OpportunityLineitems == null) {
			opp.addError('You must select an configuration product before you can set the Config Required field to true');
		}
	}
}

Hope this helps as a staring point.