I would suggest you to rethink on your design. There is no way you can get away for governor limits if you are deciding to count the records whenever you are performing an DML operation. You might solve the problem in a way to get around it but as days passes and records grow in future, you would certainly reconsider your code during that time.

Hence better change the design if it is not too late now.

Quick option:

      If you think there are not more than 500 records to count and your batch size is always 1 you can try this algoritm:

Again this is under governor limits and consumes more resources:

integer batchSize = trigger.new.size();
//query rows returned in triggers depend on batch size   
List<Id> consideredIds = new List<Id>();

integer count = 0; // variable to hold the count of records
integer appliedLimit= batchSize * 100;
//documentation suggests batchsize * 1000 but keep 100 for safer side
  List<Account> currentAccounts;

for(integer i=0; i< 5;i++){

      
      currentAccounts = new List<Account>();
      //as loop progresses records returned in previous iteration are rejected by filter
      currentAccounts = [SELECT Id 
                         FROM   Account 
                         WHERE  Custom_Field__c = 'TestValue' 
                                   AND 
                                Id NOT IN :consideredIds 
                         LIMIT  :appliedLimit
                        ];

      if(currentAccounts.size() == 0) {

         break;

      }else{

             count += currentAccounts.size();

             for(Account a: currentAccounts){

                   consideredIds.add(a.Id);

             }//inner for loop ends

       }

  }//for loop ends

I truley understand this is a dirty trick to count records but if you have run out of options you can give a try.

Risks involved in this:

1) Since SOQL query is inside for loop, and executes at max 5 times, a lot of system resources might get consumed when you do unit testing all classes(Test All)

2) if >500 records to count then you might  rethink to increase the for loop iteration. Here a balance between resources and records is in question, which may seem unstable as records grow.

3) You can say more looking at your solution design.......

As always , to count records, use a asynchrounous execution like batch or @future annotion.

Cheers..