In this post, I am not gonna explain all the things because it has a lot of parts and it takes to much space to put everything here.

So instead of explaining everything, I will try to explain the main part of the application and how you can be able to configure it in your environment.
Let's dive in.

This application is based on the OpenCV Library. OpenCV is not the only package I used some python packages also.

OpenCV Contrib Python - The original OpenCV package does not work in this app it must be opencv-contrib-python
Numpy - For the computing data
DateTime - Two identify our meal times(Breakfast, Lunch, Dinner)
MySQLdb Python Client - To Record our user data
Pyttsx3 - To convert the text to speech with the Operating System narrator
PIL - To Manipulate our Images
OS - To do system-level tasks
Zbar - To Read data from the Qr Code


When you are working with OpenCV the first thing you will face is that when you try to match a user face from the trained data and failed to recognize that person OpenCV then choose the last person as the detected one so to avoid this thing from happening we need to add a false image with the original user image which will be placed last from the original image that is gonna be trained.
So if OpenCV does not find a match from the trained data it tries to select the false data we added because the false image is added in the last place. So if OpenCV chooses the false image we can make sure the user is invalid.
The app has three main parts.

FaceDetector.py - Which is Responsible for Detecting and Saving the User Image with its id
FaceTrainer.py - Which is Responsible for Training the User data we save from FaceDetector with the False Data
mainCameraPY.py - Which is the main part of the app which detects then send the data to Cafe.py for Recognition
Cafe.py - Which is the main and last part of the app which compare the user face from the trained data and take appropriate action based on the detected data

1. Face Detector

The FaceDetector.py File is located in TrainerAndDetector Folder.
As you can see the first lines we import the libraries we will use.

import numpy as np
import cv2
import os
import shutil
import time


The below line as you can see we are declaring variable we will use inside our detector.

The first two lines are the ones responsible for detecting the face and eye of the user

In the third line we start our Camera in this case Camera 0 if You have multiple cameras you can change the value to the appropriate number

FaceDetect = cv2.CascadeClassifier('haarcascade_frontalface_default.xml')
EyeCascade = cv2.CascadeClassifier('haarcascade_eye.xml')
cap = cv2.VideoCapture(0)

WHITE = [125,30,255]
sampleNumber = 0;

userIDNum = None


This function checks if a path is exist.

We will use this to check if a user data is already exist later on.

def isPathExist(path):
    a = 0
    dir = os.path.dirname(path)
    if not os.path.exists(dir):
        os.makedirs(dir)
        a = 1
    return a


In the below code we input the User ID Number and Then we try to capture 300 photos with the user photo and save them with the user Face.

In between this we add Square to show the user face area.

You Can decrease the number of photos to be captured. in our case we used 300 for better recognition.

while aa == 0:
    while b == 0:
        os.system("cls")
        userIDNum = input("Enter The User Identification Number: ")
        
        path = "UserData/"+str(userIDNum)+"/"
        if isPathExist(path) == 1:
            break
            b = 1
        else:
            print("The User is Already Registered")
        time.sleep(1)

    print("Capturing User Face For Registration")
    os.system("cls")

    while sampleNumber < 300: 
        ret, DetectFace = cap.read()
        GrayColor = cv2.cvtColor(DetectFace, cv2.COLOR_BGR2GRAY)    
        Faces = FaceDetect.detectMultiScale(DetectFace, 1.3, 5)    
        for (x,y,w,h) in Faces:
            GrayFace = cv2.resize((GrayColor[y: y+h, x: x+w]), (110, 110))
            Eyes = EyeCascade.detectMultiScale(GrayFace)
            for (ex, ey, ew, eh) in Eyes:
                sampleNumber = sampleNumber+1;
                cv2.line(GrayColor, (x, y), (x + (w/5) ,y), WHITE, 2)
                cv2.line(GrayColor, (x+((w/5)*4), y), (x+w, y), WHITE, 2)
                cv2.line(GrayColor, (x, y), (x, y+(h/5)), WHITE, 2)
                cv2.line(GrayColor, (x+w, y), (x+w, y+(h/5)), WHITE, 2)
                cv2.line(GrayColor, (x, (y+(h/5*4))), (x, y+h), WHITE, 2)
                cv2.line(GrayColor, (x, (y+h)), (x + (w/5) ,y+h), WHITE, 2)
                cv2.line(GrayColor, (x+((w/5)*4), y+h), (x + w, y + h), WHITE, 2)
                cv2.line(GrayColor, (x+w, (y+(h/5*4))), (x+w, y+h), WHITE, 2)

                r = 280.0 / GrayColor[y:y+h, x:x+w].shape[1]
                dim = (280, int(GrayColor[y:y+h, x:x+w].shape[0] * r))
                
                # perform the actual resizing of the image and show it
                resized = cv2.resize(GrayColor[y:y+h, x:x+w], dim, interpolation = cv2.INTER_AREA)

                cv2.imwrite('UserData/'+str(userIDNum)+'/User.'+str(userIDNum)+'.'+str(sampleNumber)+'.png',resized)  
        cv2.imshow('Face and Eye Detector',GrayColor)
        cv2.waitKey(1);

After we finished Capturing the user face the next thing we need to do is to add the False data to the data we have Captured in the above step

    imgNum = 1
    while imgNum <= 200:
        if userIDNum != 1622:
            FileName1 = "False/User.9999."+str(imgNum)+".png"
        else:
            FileName1 = "FLS/User.9999."+str(imgNum)+".png"
        
        FileName2 = "UserData/"+str(userIDNum)+"/User.9999."+str(imgNum)+".png"
        shutil.copyfile(FileName1, FileName2)
        imgNum = imgNum + 1

    print("The User Face Captured Sucessfully")


2. Face Trainer

The Second Part of the app is FaceTrainer which is responsible for training the user data.

Lets see it line by line.

As always the first part of the journey is to import the packages we will be using.

import os
import cv2
import numpy as np
from PIL import Image
import MySQLdb
import datetime
import time

After we import the packages we will then initiate the LBPHFaceRecoginizer.

FaceRecognizer = cv2.face.LBPHFaceRecognizer_create()

The below line checks if we have the already trained the selected user data.

def isPathExist(path,UserIDNum):
    a = 0
    dir = os.path.dirname(path)
    if os.path.exists(dir):
        tPath = "../TrainingData/"+str(UserIDNum)+"/"
        tDir = os.path.dirname(tPath)
        if not os.path.exists(tDir):
            os.makedirs(tDir)
            a = 1
        else:
            a = 2
    return a


The below code will get all the images we saved from the FaceDetector with the selected User ID Number

def getUserImage(UserDataPath) :
    UserImagePath = [os.path.join(UserDataPath, f) for f in os.listdir(UserDataPath)]
    UserFaces = []
    UserIDs = []
    for ImagePath in UserImagePath :
        UserFaceImage = Image.open(ImagePath).convert('L')
        FaceNp = np.array(UserFaceImage, 'uint8')
        ID = int(os.path.split(ImagePath)[-1].split('.')[1])
        UserFaces.append(FaceNp)
        UserIDs.append(ID)
        cv2.waitKey(10)
    return np.array(UserIDs), UserFaces


The next line will Accept the user Qr Code and check if the user is registered or the user data is trained.

If the user does have not a photo the system notifies her/him to take a photo and come back.

Finally if all the verification has passes the FaceRecognizer.train() function trains the data and save it to TrainingData/User ID/User ID.yml Yaml File.

while conReg == 1:
    while True:
        #os.system("cls")
        global UserDataPath
        global UserIDNum
        UserIDNum = input("Enter The User QRCode Number: ")
        UserDataPath = 'UserData/'+str(UserIDNum)+"/"
        
        if isPathExist(UserDataPath,UserIDNum) == 1:
            break
        elif isPathExist(UserDataPath,UserIDNum) == 2:
            print("The User Is Already Registered")
        else:
            print("Please Take User A Photo Before Training")
        time.sleep(1)

    FName = str(raw_input("Enter The First Name  : "))
    MName = str(raw_input("Enter The Middle Name : "))
    LName = str(raw_input("Enter Last Name       : "))

    UserIDs, UserFaces = getUserImage(UserDataPath)
    print("Please Wait Until The User Registered")

    try:
        FaceRecognizer.train(UserFaces, np.array(UserIDs))
        FaceRecognizer.save('../TrainingData/'+str(UserIDNum)+'/'+str(UserIDNum)+'.yml')
        print("Sucessfully Saved Trained Data")

        db = MySQLdb.connect("localhost","root","","cafepro")
        cursor = db.cursor()
        now = datetime.datetime.now()

        sql = "INSERT INTO cafeuser(QrNum,FName,MName,LName,userMonth,userDate,Breakfast,Lunch,Dinner,Disabled) VALUES("+str(UserIDNum)+",'"+FName+"','"+MName+"','"+LName+"',"+str(now.month)+","+str(now.day)+",0,0,0,0)"
        cursor.execute(sql)
        db.commit()  
        cursor.close()
        print("The User Is Already Registered")
        print("He Can Use Cafe From Now")
    except :
        print("Error Occured Try To Register Again")

3. mainCameraPY

As always the first part of the journey is to import the packages we will be using.

import numpy as np
import cv2
import datetime
import MySQLdb
import zbar
from PIL import Image
import os
import pyttsx3

After we imported our packages we will declare our variables.

In line 10 and 11 we have the haarcascade face and eye contour.

In line 12 we have the LBPHFaceRecognizer instance.

In line 14 we have our database instance.

In line 17 we have loaded the OpenCV camera instance and so on.

FaceDetect = cv2.CascadeClassifier('HaarCascade/haarcascade_frontalface_default.xml')    # Load The HAAR CASCADE Face Detecter
EyeCascade = cv2.CascadeClassifier('HaarCascade/haarcascade_eye.xml')                    # Load The HAAR CASCADe Eye Detector
FaceRecognizer = cv2.face.LBPHFaceRecognizer_create()                                    # Load The OpenCV Face Recognizer

db = MySQLdb.connect("localhost","root","","cafepro")
isDetected = False                                       
WHITE = [255, 255, 255]
capVideo = cv2.VideoCapture(0)                  # Load The OpenCV Video Streamer
font = cv2.FONT_HERSHEY_SIMPLEX         

NormalGrayFace = None
NewBlackImage = cv2.imread("BlackImage.jpg")    # Load A Background Image For The Image Binder

now = datetime.datetime.now()

cursor = db.cursor()
sql = "SELECT * FROM mealtime WHERE mealMonth="+str(now.month)+" AND mealDate="+str(now.day)
cursor.execute(sql)
db.commit()
row = cursor.fetchone()
cursor.close()

BreakfastEaten = 0    # Number of Users Eaten Their BreakFast ToDay
LunchEaten = 0        # Number of Users Eaten Their Lunch ToDay
DinnerEaten = 0       # Number of Users Eaten Their Dinner ToDay

if row is not None:
    BreakfastEaten = int(row[2])                                             
    LunchEaten = int(row[3])                                                     
    DinnerEaten = int(row[4])                                                    
  

The below function is responsible for displaying information to the user like adjusting their face.

if a face is detected it sets the variable isDetected to true.

def readCamera():
    global blackImage
    global isDetected
    global GrayFace
    global GrayColorImage
    global FullChImage

    blackImage = NewBlackImage
    now1 = datetime.datetime.now()                                      # Get The Current Date And Time        
    
    isDetected = False
    ret, DetectFace = capVideo.read()                                   # Read Video From The Video Streamer
    GrayColor = cv2.cvtColor(DetectFace, cv2.COLOR_BGR2GRAY)            # Convert The Image To Gray Image
    FullChImage = GrayColor                                     
    Faces = FaceDetect.detectMultiScale(GrayColor, 1.3, 5)              # Find Face From The Gray Image
    
    for (x,y,w,h) in Faces:                                             # If Face is Detected 
        GrayFace = cv2.resize((GrayColor[y: y+h, x: x+w]), (110, 110))  # Crop Only The Face Part From The Image
        GrayColorImage = GrayColor[y:y+h, x:x+w]
        Eyes = EyeCascade.detectMultiScale(GrayFace)                    # Find Eyes From The Detected Faces
        for (ex, ey, ew, eh) in Eyes:                                   # If Two Eyes Found From The Detected Face
            isDetected = True
            NAME = "Face Detected"
            drawRectangle(DetectFace,x,y,w,h,NAME)                      # Draw A Rectangle In To The Detected Face        
            
    cv2.putText(DetectFace,'Position Your Face Correctly',(10,50),font, 0.5,(70,0,100),2)
    rows = 480
    cols = 640
    channels = 3

    roi = blackImage[0:rows, 0:cols ]
    ret, mask = cv2.threshold(GrayColor, 0, 255, cv2.THRESH_BINARY)
    mask_inv = cv2.bitwise_not(mask)

    img1_bg = cv2.bitwise_and(roi,roi,mask = mask_inv)

    img2_fg = cv2.bitwise_and(DetectFace,DetectFace,mask = mask)

    dst = cv2.add(img1_bg,img2_fg)
    blackImage[0:rows, 0:cols ] = dst                                         # Bind The Streamed Image With The Background Image Loaded Above

    CurrTime = str(now1.hour)+":"+str(now1.minute)+":"+str(now1.second)       
    cv2.putText(blackImage,CurrTime,(500,50), font, 0.5,(70,0,100),2)         # Bind The Current Date and Time With The Image

    return blackImage                                                         # Return The Whole Image Bind With The Streamed Image and Current Date and Time


The Next line is responsible for checking if the user face with the associated qr code is correct or not by resizing the image to a specified width and height and based on the status code returned this function then calls another function.

1. Cheater -> if the face does not match with the data associated with the QrCode.

2. CheckMeal -> if the user match then check if he/she has eaten their meal.

def checkUser(QRCode):
    userDir = "TrainingData/"+QRCode+"/"+QRCode+".yml"                          
    try:                                                   
        FaceRecognizer.read(userDir)   
        r = 280.0 / GrayColorImage.shape[1]
        dim = (280, int(GrayColorImage.shape[0] * r))
                
        resized = cv2.resize(GrayColorImage, dim, interpolation = cv2.INTER_AREA)
        userQRCode, conf = FaceRecognizer.predict(resized)          # Predict The The Detected Face From The DataSet

        if userQRCode == int(QRCode):                               # If The QrCode Entered is Associated With The DataSet is Correct
            isEaten = checkMeal(QRCode)                             # WellCome User and Update User To Ate State
        else :                                                      # Else The Face is Not Matched With The DataSet
            useOtherMealCard(QRCode)                                # The User is Using Other Person MealCard So Enter User To The Cheaters List 
            isEaten = 3

        return isEaten
    except:                                                                                                      #Catch If The User is Not Registered
        return 4

The below lines are to check if our Meal Time and Meal Type which will be used to make the system available for the user and also to make database modification based on the meal type and meal time.

And we have the getUserImage function which will show which user is being checked.

def mealTimeCheck():
    now = datetime.datetime.now()
    mealTime = 0

    if now.hour >= 16 and now.hour < 18: 
        mealTime = 6
    elif now.hour >= 0 and now.hour <= 2:
        mealTime = 7
    elif now.hour >= 4 and now.hour < 8:
        mealTime = 8
    
    return mealTime

def mealTypeCheck():
    now = datetime.datetime.now()
    mealType = ""

    if now.hour >= 16 and now.hour < 18: 
        mealType = "Breakfast"
    elif now.hour >= 0 and now.hour <= 2:
        mealType = "Lunch"
    elif now.hour >= 4 and now.hour < 8:
        mealType = "Dinner"
    
    return mealType


def getUserImage(QRCode):
    userDir = "UserImages/"+QRCode+"/"+QRCode+".png"
    userImage = cv2.imread(userDir,0)
    return userImage

The next line is to check which meal type and based up on the response it will call two Function to make state changes.

1. triedMoreTime -> which is a state when a user tries to eat more than one time in the same meal period.

2. updateToAte -> which is a state when a user is not yet eaten their meal so it will update the database to eaten state.


def checkMeal(QRCode):
    cursor = db.cursor()
    now = datetime.datetime.now()

    sql = "SELECT * FROM cafeuser WHERE QrNum="+QRCode
    cursor.execute(sql)
    db.commit()  
    row = cursor.fetchone()
    cursor.close()

    mealTime = mealTimeCheck()

    if row is not None:
        if row[9] == 0:
            if row[4] == now.month and row[5] == now.day:
                if row[mealTime] == 1:
                    triedMoreTimes(QRCode)
                    isEaten = 2
                else :
                    updateToAte(QRCode)
                    isEaten = 1
        else:
            isEaten = 5
    else :
        if row[9] == 0:
            updateToAte(QRCode)
            isEaten = 1
        else:
            isEaten = 5
    
    return isEaten

The below line we have three function we have discussed above.

1. triedMoreTimes

2. useOtherMealCard

3. updateToAte

def triedMoreTimes(QRCode):
    now = datetime.datetime.now()
    mealType = mealTypeCheck()
    mealTime = mealTimeCheck()
    
    chNum = 1
    chDir = "CheaterImage/"+str(now.year)+str(now.month)+str(now.day)

    cursor = db.cursor()
    selectFromCheater = "SELECT * FROM cheateruser WHERE userMonth="+str(now.month)+" AND userDate="+str(now.day)
    cursor.execute(selectFromCheater)
    db.commit()
    rowcount = cursor.rowcount
    cursor.close()

    if not os.path.exists(chDir):
        os.makedirs(chDir)

    chNum = int(rowcount) + 1

    imgName = str(now.year)+str(now.month)+str(now.day)+str(chNum)
    imgDir = chDir+"/"+imgName+".png"

    cv2.imwrite(imgDir,FullChImage)

    mealType2, mealType3 = NotMealType(mealType)

    cursor = db.cursor()
    TriedUser = "INSERT INTO cheateruser(QrNum,userMonth,userDate,ImageName,"+mealType+","+mealType2+","+mealType3+",Tried,OtherMeal,Disabled) VALUES("+QRCode+","+str(now.month)+","+str(now.day)+","+imgName+",1,0,0,1,0,0)"
    cursor.execute(TriedUser)
    db.commit()
    cursor.close()

def useOtherMealCard(QRCode):
    now = datetime.datetime.now()

    mealType = mealTypeCheck()
    mealTime = mealTimeCheck()
    
    chNum = 1
    chDir = "CheaterImage/"+str(now.year)+str(now.month)+str(now.day)

    cursor = db.cursor()
    selectFromCheater = "SELECT * FROM cheateruser WHERE userMonth="+str(now.month)+" AND userDate="+str(now.day)
    cursor.execute(selectFromCheater)
    db.commit()
    rowcount = cursor.rowcount
    cursor.close()

    if not os.path.exists(chDir):
        os.makedirs(chDir)

    chNum = int(rowcount) + 1

    imgName = str(now.year)+str(now.month)+str(now.day)+str(chNum)
    imgDir = chDir+"/"+imgName+".png"

    cv2.imwrite(imgDir,FullChImage)

    mealType2, mealType3 = NotMealType(mealType)

    cursor = db.cursor()
    OtherPersonMeal = "INSERT INTO cheateruser(QrNum,userMonth,userDate,ImageName,"+mealType+","+mealType2+","+mealType3+",Tried,OtherMeal,Disabled) VALUES("+QRCode+","+str(now.month)+","+str(now.day)+","+imgName+",1,0,0,0,1,0)"
    cursor.execute(OtherPersonMeal)
    db.commit()
    cursor.close()



def updateToAte(QRCode):
    now = datetime.datetime.now()

    mealType = mealTypeCheck()
    mealTime = mealTimeCheck()

    cursor = db.cursor()
    UpdateCafeUser = "UPDATE cafeuser SET userMonth="+str(now.month)+",userDate="+str(now.day)+","+mealType+"=1 WHERE QrNum="+QRCode
    cursor.execute(UpdateCafeUser)
    db.commit()
    cursor.close()

    cursor = db.cursor()
    selectFromMealDate = "SELECT * FROM mealtime WHERE mealMonth="+str(now.month)+" AND mealDate="+str(now.day)
    cursor.execute(selectFromMealDate)
    db.commit()  
    row = cursor.fetchone()
    cursor.close()

    mealTime = mealTime - 4 

    numUser = 0
    if row is not None:
        numUser = row[mealTime] + 1

        cursor = db.cursor()
        UpdateMealDate = "UPDATE mealtime SET "+mealType+"="+str(numUser)+" WHERE mealMonth="+str(now.month)+" AND mealDate="+str(now.day)
        cursor.execute(UpdateMealDate)
        db.commit()
        cursor.close()
    else:
        cursor = db.cursor()
        UpdateMealDate = "INSERT INTO mealtime (mealMonth, mealDate, Breakfast, Lunch, Dinner) Values("+str(now.month)+","+str(now.day)+",0,0,0)"
        cursor.execute(UpdateMealDate)
        db.commit()
        cursor.close()

        cursor = db.cursor()
        UpdateMealDate = "UPDATE mealtime SET "+mealType+"=1 WHERE mealMonth="+str(now.month)+" AND mealDate="+str(now.day)
        cursor.execute(UpdateMealDate)
        db.commit()
        cursor.close()

    BreakfastEaten = row[2] + 1
    LunchEaten = row[3] + 1
    DinnerEaten = row[4] + 1

4. Cafe.py

Also in this part we will import all the packages and also mainCameraPY we have discussed earlier.
import mainCameraPY as MC
import cv2
import zbar
import datetime
from PIL import Image
import time
import os
import pyttsx3
import MySQLdb

In the first line we clear the screen.

In the next line we have OpenCV camera instance.

In the forth and fifth line we initiated the pyttsx3 which is responsible for converting text to speech and set the default voice from a window machine.

In line seven we have initiate database connection.

os.system("cls")
capVideo = cv2.VideoCapture(0)

engine = pyttsx3.init()
engine.setProperty('voice', 'HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Speech\Voices\Tokens\TTS_MS_EN-US_DAVID_11.0')

db = MySQLdb.connect("localhost","root","","cafepro")
now = datetime.datetime.now()

Finally we have the MealApp function we will call the readCamera() function from the mainCamerPY we have imported before and show the camera view for the user and also accept the QrCode from then decry-pt it and send it to checkUser from mainCameraPy.

def MealApp():
    global QRCode
    global isEaten

    while True:
        isEaten = 0
        now = datetime.datetime.now()
        if (now.hour >= 16 and now.hour < 18) or (now.hour >= 0 and now.hour <= 5) or (now.hour >= 4 and now.hour < 8):
            isQrDetected = False
            frame = MC.readCamera()
	    cv2.rectangle(frame,(1200,500),(500,730),(100,70,0),-2)
	    cv2.putText(frame,"Current Date : "+str(now.year)+"-"+str(now.month)+"-"+str(now.day),(550,550),MC.font, 1,(70,0,100),2)
	    cv2.putText(frame,"BreakFast Eaten User : "+str(MC.BreakfastEaten),(600,600),MC.font, 0.8,(255, 255, 255),2)
	    cv2.putText(frame,"    Lunch Eaten User : "+str(MC.LunchEaten),(600,630),MC.font, 0.8,(255, 255, 255),2)
	    cv2.putText(frame,"    Dinner Eaten User : "+str(MC.DinnerEaten),(600,660),MC.font, 0.8,(255, 255, 255),2)
            cv2.imshow("Face Recognition Cafe Monitering System",frame)
            cv2.waitKey(1)


            if MC.isDetected == True:
                i = 0;
                while i < 10:
                    frame = MC.readCamera()
                    cv2.imshow("Face Recognition Cafe Monitering System",frame)
                    cv2.waitKey(1)
                    i = i + 1
                    blackImage = MC.blackImage
		cv2.rectangle(blackImage,(1200,500),(500,730),(100,70,0),-2)
		cv2.putText(frame,"Current Date : "+str(now.year)+"-"+str(now.month)+"-"+str(now.day),(550,550),MC.font, 1,(70,0,100),2)
		cv2.putText(blackImage,"BreakFast Eaten User : "+str(MC.BreakfastEaten),(600,600),MC.font, 0.8,(255, 255, 255),2)
		cv2.putText(blackImage,"    Lunch Eaten User : "+str(MC.LunchEaten),(600,630),MC.font, 0.8,(255, 255, 255),2)
		cv2.putText(blackImage,"    Dinner Eaten User : "+str(MC.DinnerEaten),(600,660),MC.font, 0.8,(255, 255, 255),2)

                QRDetect = False
                while QRDetect == False:
                    now1 = datetime.datetime.now()   
                    ret, img = capVideo.read()
                    gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
                    image = Image.fromarray(gray)
                    width, height = image.size
                    zbar_image = zbar.Image(width, height, 'Y800', image.tobytes())

                    scanner = zbar.ImageScanner()
                    scanner.scan(zbar_image)
                    for decoded in zbar_image:
                        if decoded.data is not None:
                            QRCode = decoded.data
                            QRDetect = True


                    rows = 480
                    cols = 640
                    channels = 3

                    roi = blackImage[0:rows, 0:cols ]
                    ret, mask = cv2.threshold(gray, 0, 255, cv2.THRESH_BINARY)
                    mask_inv = cv2.bitwise_not(mask)

                    img1_bg = cv2.bitwise_and(roi,roi,mask = mask_inv)
                    img2_fg = cv2.bitwise_and(img,img,mask = mask)

                    dst = cv2.add(img1_bg,img2_fg)
                    blackImage[0:rows, 725:cols+725 ] = dst


                    CurrTime = str(now1.hour)+":"+str(now1.minute)+":"+str(now1.second)
                    cv2.putText(blackImage,'Position Your QRCode Correctly',(750,50),MC.font, 0.5,(70,0,100),2)
                    cv2.putText(blackImage,CurrTime,(1250,50), MC.font, 0.5,(70,0,100),2)
                    
                    if QRDetect == True:
                        try:
                            if int(QRCode) <= 0 and int(QRCode) >= 9999:
                                QRDetect = False
                                engine.say('You Entered Invalid Meal Card Code')
                                engine.runAndWait()
                            else:
                                engine.say('Meal Card Detected')
                                engine.runAndWait()
                        except:
                            QRDetect = False
                            engine.say('You Entered Invalid Meal Card Code')
                            engine.runAndWait()
                            
                    cv2.imshow("Face Recognition Cafe Monitering System",blackImage)
                    cv2.waitKey(1)


                cv2.imshow("Face Recognition Cafe Monitering System",blackImage)
                cv2.waitKey(1)

                cv2.putText(blackImage,'Position Your QRCode Correctly('+ QRCode +')',(750,50),MC.font, 0.5,(70,0,100),2)
                isEaten = MC.checkUser(QRCode)

                if isEaten == 1 or isEaten == 2:
                    cursor = db.cursor()
                    sql = "SELECT * FROM cafeuser WHERE QrNum="+QRCode
                    cursor.execute(sql)
                    db.commit()  
                    row = cursor.fetchone()
                    cursor.close()

                    mealType = MC.mealTypeCheck()

                    if isEaten == 1:
                        engine.say('Hello '+row[1]+' '+row[2]+' Have A Good '+mealType)
                        engine.runAndWait()

                    else:
                        engine.say(row[1]+' '+row[2]+' You Are Cheating')
                        engine.runAndWait()

                elif isEaten == 3:
                    engine.say('You Have Stolen Other Persons Meal Card')
                    engine.runAndWait()

                elif isEaten == 4:
                    engine.say('You Are Using Invalid Meal Card')
                    engine.runAndWait()

                elif isEaten == 5:
                    engine.say('You Meal Card is Disabled Please ReEnabled it Again')
                    engine.runAndWait()

        else :
            ImageNotOpened = cv2.imread("BlackImage.jpg")
            CurrTime = "Cafe is Closed "
            cv2.putText(ImageNotOpened,CurrTime,(300,400), MC.font, 2,(25,255,255),2)
            CurrTime = "Current Time "+str(now.hour)+":"+str(now.minute)+":"+str(now.second)
            cv2.putText(ImageNotOpened,CurrTime,(400,500), MC.font, 2,(25,255,255),2)

            MC.readCamera()
            if MC.isDetected == True:
                engine.say('Dear User The Cafe is Still Closed')
                engine.runAndWait()

            cv2.imshow("Face Recognition Cafe Monitering System",ImageNotOpened)
            cv2.waitKey(1)

Finally we call run the app by calling the MealApp() function to run the app.

MealApp()