BASED AUTOMATIC PERSONALITY RECOGNITION USED IN ASYNCHRONOUS VIDEO INTERVIEWS OF STRESS DETECTION US

Page 1

InternationalJournalofComputer ScienceandEngineering(IJCSE)

ISSN(P):2278–9960;ISSN(E):2278–9979

Vol.11,Issue2,Jul–Dec2022;1–8 ©IASET

BASEDAUTOMATICPERSONALITYRECOGNITIONUSEDINASYNCHRONOUS VIDEOINTERVIEWSOFSTRESSDETECTIONUSINGFACEIMAGESANDFACIAL LANDMARKBYUSINGTHECONVOLUTIONNEURALNETWORK(CNN)ALGORITHM

PriyankaH1&BLJayakumar2

1ResearchScholar,DepartmentofComputerScience&Engineering,SEACollegeofEngineering&Technology, KRPuram,Bengaluru,India

2AssistantProfessor,DepartmentofComputerScience&Engineering,S.E.ACollegeofEngineering&Technology, K.RPuram,Bengaluru,India

ABSTRACT

Withthehelpoffacephotosandfaciallandmarks,wesuggestastressrecognitionalgorithminthiswork.Adevicefor gatheringthenecessarydataisneededalongtheeventofstressdetectionutilisinganaturalorbiologicalsignalor thermalpicture,thusbeingimportantareaforresearch.Toaddressthisflaw,weputforthanalgorithmthatcanidentifya person'sbehaviourfromstillvideosorphotostakenusinganormalcamera,includingcreatingin-depthneuralnetwork, usesfacialidentificationsinfusedalongbenefitingthatwhensomeoneisbeingstressed,theireye,mouth,andhead movementsdifferalonghowtheynormallybehave.Likewise,byidentifyingacandidate'sbehaviourduringanonline interview,wecandeterminewhetherornottheyarequalified.Thesuggestedalgorithmrecognisesbehaviourmore accurately,accordingtoexperimentaldata.

KEYWORDS:C++,Python,Java,ConvolutionalNeuralNetwork(CNN),PersonalityRecognition,OpenCV,HAAR CascadeandMatLab,OpenCV,Espeak,Xming&Putty

ArticleHistory

Received:06Jul2022|Revised:07Jul2022|Accepted:07Jul2022

1.INTRODUCTION

Inmanysituationswheremoresecurityorpersonaldataaboutthepersonisneeded,humanemotiondetectionisused.We couldneedtosetupasecondlayerofprotectionwhere,inadditiontotheface,theemotionisalsodetected.Thiscanbe consideredasthesecondstageafterfacedetection.Asystemiscurrentlybeingdevelopedtodetectwhenauserisunder stressandtoprovidefeedbackintheaimofreducingstresswhenunderstress,asmodernindividualsexperience tremendouslevelsofstress.Additionally,wesuggestedthatthestudent'semotionsbeacknowledgedandthattheworried teacherreceiveanupdate.Focusingonknowingclassesforstudentsincludingtime-tablebeingsharedamongstthem.

Intheseinvestigations,characteristicsfrombiosignalssuchtheelectrocardiogram,electrodermalactivity, respiration,galvanicskinresponse,andheartratevariabilitywereextractedandusedtoexpressstress.Furthermore,alot ofthemmadeuseoftraditionalclassifierslikeSupportVectorMachine,LinearDiscriminantAnalysis,Adaboost,andKNearestNeighbor[1].

www.iaset.us

editor@iaset.us

Alongthevicinityofrecognisingstresses,numerousmethodsthrubio-signalsbeingevaluated,thoughameasure andalongbio-signalling,anend-usermayexperiencerejectionsduetotoolsappliedinmeasuringbio-signallingthusbeing rootedalongthebody.Thusnumerousobservationsalongstress-recognitionthruthermalimagingweredone,thusevena demeritalongcomplicationsinrecognizingstresssimplyalongeverydaylifeduetounrecognisablewithoutanytoolsof thermalimaging,parallellyintheventofstress-recognitionobservationsthrucommonimaging,popularevaluations adaptedbeingconsiderablyfeaturedassimple.

Inthisarticle,weprovideatechniqueforidentifyingstressthatinvolvesextractinghigh-dimensionalfeatures fromfacephotostakenwitharegularcamera.Additionally,weusetheplacementoffacelandmarksthatsignificantlyvary whenunderstressinordertolearnmoreeffectivefeatures.

2.RELATEDWORK

Physiologicalparameterevaluation,bodilyfluidmeasurement,andthepapermethod(self-report)aresomeofthe behaviouranalysistechniquesnowinuse.Inthepaperapproach,individualsaregivenamultiple-choicequestionnaireto complete,andeachresponseisassignedaspecificscore.Aftertherespondentcompletesthequestionnaire,theresultsof eachchoicewillbeaddeduptoproduceafinalscorethatrepresentstheindividual'slevelofconduct.Fortheidentification ofthestresshormone,Cortical,bodyfluidtestssuchasbloodorsalivaareperformed.Thesetwoapproachesare ineffectivefortrackingstressovertime.Monitoringandanalysingphysiologicaldatacangiveoneausefulunderstanding oftheirhealth.

BehaviorRecognitionusingBio-Signals

Becausetheydisplaythebody'smostsensitivealterationsandenabletheidentificationofchangesinthebodythatarenot indicatedbyfaceandbehaviour,bio-signalswereusedinearlyresearchonstressrecognition.Intheseinvestigations, characteristicsfrombio-signalssuchtheelectrocardiogram,electrodermalactivity,respiration,galvanicskinresponse,and heartratevariabilitywereextractedandusedtoexpressstress.AndalotofthemmadeuseoftraditionalclassifierslikeKNearestNeighbor,AdaBoost,SupportVectorMachine,andlineardiscriminantanalysis.

BehaviorRecognitionUsing

Thermalpicturemanystudieshavebeendonetodetectthechangebyusingthethermalimagingtodetectthechange becausewhenapersonisstressed,thebloodflowandtemperatureofthefaceincrease.Thisresearchusedavarietyof techniquestoidentifystress,includingdirectlyextractingcharacteristicsfromthermalpicturesandextractingfeatureslike respirationrate,blinkfrequency,skintemperature,andbloodflowfromthermalimages.

BehaviorRecognitionusingGeneralImage

Eye,lip,andheadmovementschangewhenapersonisstressedcomparedtonormalcircumstances,andresearchonstress recognitionusingcommonimageryisalsobeingdone.Theseresearchemployedavarietyoftechniquestoidentifystress, includingtheextractionofhand-craftedfeaturesfromthenose,mouth,andeyeregionsaswellastheuseofeyesize,lip motions,andheadmovementsasfeatures.

2 PriyankaH&BLJayakumar
ImpactFactor(JCC):8.5226 NASRating:3.17

BasedAutomaticPersonalityRecognitionUsedinAsynchronousVideoInterviewsofStressDetectionAlgorithm 3

UsingFaceImagesandFacialLandmarkbyUsingtheConvolutionNeuralNetwork(CNN)

3.LITERATUREREVIEW

HumanAttitude-RecognitionApplicationsBasedonFacialFeaturethroughFaceDetection

Author:ArdinintyaSetyadiDivaet.al,

Publishedin:2015InternationalElectronicsSymposium(IES)

Abstract

Human-psychology,exploredalong4basicpersonalitieslikesanguine,choleric,melancholic,andphlegmatic.Basicmode ofknowinghumanfundamentalpersonalityisbytesting,thusbeingGraphotest(handwritingtest).Currentstudyitsbeing executedondetectinghumanfundamentalpersonalitythrucollectivefeaturedfaces:theeyes,lips,andnose(priortotest), thusgotthrureceivedimagesoffaces,gapamongstthecornersoftheeyes,ratiosamongstmouthwidthandnose,ratiosof widthamongsteyes,lipthicknessbeingextractingfeatures,thruANN:artificialneuralnetworks(back-propagation),also relayingonextractingsuchfeatures,thefundamentalpersonalitybeingidentifiedConsideringpracticalfindings,system detectshumanfundamentalpersonalityalongsimilarinputimagingdataalongtrainingaverageratios85.5%.The identificationfindingsvariousinputimagingdataalongtrainingbeingweighedas42.5%average,situationsdemandsthru identificationofpersonalityalongcholeric,phlegmaticreadslessenedabout50%ratios.

ADVANTAGE

 Theyhaveimplemented2typesoftesting,oneonthesamephotoandanotheronthedifferentphoto.

DISADVANTAGE

 Testingaccuracybeingminimallyabout50%alongfewfeaturesonalimitedoccasions

SentimentClassificationandPersonalityDetectionviaGalvanicSkinResponseBasedonDeepLearningModels

Author:TaoHong;XiaoSun;FangTian;FujiRen

PublishedIn:20195thInternationalConferenceonBigDataComputingandCommunications(BIGCOM)

Abstract

Sentimentandpersonalityhaveasignificantinfluenceonhowwethink,create,andmakedecisionsinourdailylives. Manymethodshavebeenputforthtoautomaticallydetectusers'sentimentinspeechandimage.Forsomecircumstances, suchasinterviewsandpolygraphs,beingabletoaccuratelyforecastaperson'semotionsandpersonalitytraitscanbe helpful.Inthiswork,severalmodels,suchasJointLearningModelofConvolutionalNeuralNetworkandmemoryalong long-Short-Term,alsospatiotemporalhybridmodelswereprojectedtoknowautomaticallyaboutgalvanicskinresponse (GSR),alsovideoclipsratingagainstsimilartosentimentsrecognition,personalitydetectionfacts,matchingalong projectedmodels,alsostate-of-the-artmodelsalongsimilarworks.Thepracticalfindingspresentedagainstpredominant findingsalongsentimentrecognition,alsopersonalitydetectionbeingbenefittedalongprecision,recall,andF1score.

ADVANTAGE

 MatchedagainstothersimilartrialsadaptingGalvanicSkinresponse(GSR)signalalongemotionalcategorisation ofpersonalitydetection,practicalfindingsareenhanced.

www.iaset.us editor@iaset.us

DISADVANTAGE

 Thecategorisationalongsurprise,happinessbeingatoughfact

StudyonDeterminingtheBig-FivePersonalityTraitsofanIndividualBasedOnFacialExpressions

PublishedIn:2015E-HealthandBioengineeringConference(EHB)

Abstract

Accordingtopreviousstudies,thereisanincreasingdemandforinformationaboutpeople'spersonalitiesandbehavioursin areaslikecareerdevelopmentandcounselling,individualisedhealthcare,mentaldisorderdiagnosis,andtheearly detectionofphysicalillnesseswithpersonalityshiftsymptoms.TheBig-Fivepersonalitytypescancurrentlyonlybe identifiedbycompletingaquestionnaire,whichtakesanunreasonableamountoftimeandcannotbeusedfrequently.Our researchintendstodevelopacutting-edge,non-invasivemethodforidentifyingtheBig-Fivepersonalitytraitsbasedon facialfeaturescollectedwiththehelpoftheFacialActionCodingSystem.Thefindingsindicatearelationshipbetweena person'spersonalityqualitiesandtheFACSactionunitspresentinfacefeaturesattheirhighestintensitiesAdditionally, comparedtotherequiredforcompletingaquestionnaire,thesystemdevelopeddeliversover75%accuracyofthe predictionopennesstoexperience,aswellastraitanxietyandextraversion,anditispractical,givingfindingsinlessthan3 minutes.

ADVANTAGES

 Approximatetimetakenbythesystemtoprovideresultswasroughly45secforvideosequenceswithincluded emotions.

DISADVANTAGE

 Unabletoaccuratelydetermineconscientiousnessandagreeablenesswhicharepartofpersonalitytraits.

4.IMPLEMENTATION

I.Proposedalgorithm

inthispartweprovideanapproachtoenhancetheperformanceofstressrecognitionoverallstructureinthesuggested approachstressrecognitionbeginswithfacepictureandfaciallandmarkdetectioninordertodetectfacesmorecorrectly weemployadeeplearningtechniquethatusesthreenetworkssequentiallyweemployacustomapproachthatemploysa cascadetypeofextractedfeaturesalongsudden-femalso,theregressiontreeclassifieralongrecognisingface identificationsgraphicbelowvisualisescollectiveframeworksflowchart

ImpactFactor(JCC):8.5226

NASRating:3.17

4 PriyankaH&BLJayakumar
Figure1.

BasedAutomaticPersonalityRecognitionUsedinAsynchronousVideoInterviewsofStressDetectionAlgorithm 5

UsingFaceImagesandFacialLandmarkbyUsingtheConvolutionNeuralNetwork(CNN)

Alongtheestimatednetwork,thefacialimaging,identifiedpreviouslyisinputtooutputstressrecognition findings.Thestructureoftheestimatednetworkisvisualisedalongfig.

Progressednetwork,adaptedshortcutassigning,bottleneckarchitecturetoenhanceneuralnetworkstructure. Throughthisshortcutassigningtotheneuralnetworkstructuresupportedalongnumerouslayers,encouragesin simplifyinglearningmethods,alosexplainslearningdirectionsMakingitsuccessfulinsimplyenhancingthein-depth neuralnetwork,alsotoprogressaccuracyalongprogresseddepth.Thruimplyingbottleneckarchitecture,thequantumof internalfactorsbeinglessenedalongelevatingquantumoffeaturemaps,whichelevatesfunctionality,declinesquantumof computation.

II.Input/OutputDesign

Systemdesignshowstheoveralldesignofsystem.Inthissectionwediscussindetailthedesignaspectsofthesystem:

ImageCapture

Staticphotosorimagesequencesareutilisedtodetectfaceexpressions.Acameracanrecordpicturesoffaces.Face recognition

Inordertoidentifyfacialimages,facedetectionishelpful.Facedetectionisdoneinthetrainingdatasetusingthe OpencvimplementationoftheHaarclassifierVoila-Jonesfacedetector.Thevalueofafeaturewithahaar-likestructureis thedifferenceinthesumofthepixelvaluesintheblackandwhiteregions,anditencodesthevariationinaverageintensity indifferentregionsoftheimage[6].Imagepreparation

Noiseiseliminatedduringimagepre-processing,andbrightnessandpixelpositionvariationsarenormalised.

ColorNormalization\s

HistogramNormalization

ExtractionofFeatures

Thechoiceofthefeaturevectorinapatternclassificationissueiscrucial.Afterpre-processing,thefacialimageisusedto extractthekeyfeatures.Scale,attitude,translation,andfluctuationsinilluminationlevelaresomeofthefundamental issueswithimageclassification[6].ClassificationClassificationisusedtoreducethehighdimensionalityofdatathatwas obtainedusingthefeatureextractionmethod.Convolutionalneuralnetworkalgorithmwillbeusedtoclassifyobjects,and featuresshouldtakedistinctvaluesforobjectsbelongingtodifferentclasses.

www.iaset.us editor@iaset.us

Figure2. Database Imagesof different Face detection from Image Extarct Emotion
Prediction Results
Facialpoint extraction fromImage Machine Learning Algorithm Transformed Facialpoint asfeatures

III.DataFlowDiagram

SequenceDiagram

SequenceDiagram

Inasequencediagram,objectconnectionsarearrangedintemporalorder.Itillustratestheclassesandobjectsinvolvedin thesituationaswellasthemessageflowthatmustoccurfortheobjectstofunctionasintended.Sequencediagramsand usecaserealisationsarefrequentlyconnectedinthelogicalviewofthesystemunderdevelopment.Othernamesfor sequencediagramsareeventdiagramsandeventscenarios.

ActivityDiagram

Thesystem'sdynamicfeaturesaredescribedintheactivitydiagram,aUMLdiagram.Inactuality,aflowchartcontrols howeacheventproceeds.Thesystem'soperationcanbecharacterisedastheevent.Thecontrolflowmustbefollowed betweentasks.

ImpactFactor(JCC):8.5226

NASRating:3.17

6
PriyankaH&BLJayakumar Figure3. Figure4. Figure5. PriyankaH&BLJayakumar Figure3. Figure4.

UsingFaceImagesandFacialLandmarkbyUsingtheConvolutionNeuralNetwork(CNN)

5.REQUIRMENTS

A.Functionalrequirements

Asystem'sfunctionalrequirementsspecifywhatthesystemshouldbeabletoachieve.Thesespecificationsaredetermined bythekindofsoftwarebeingcreatedandtheintendedaudience.

 CreatingWebbasedfunctionsforTraitanalysis.

 Designing/formulatingquestionsforanalysis,visualisingtousersequentially

 Capturingfacialimagingthruansweringtoquestions.

 Analyzingtraitsalongimaging

 Visualisingclarifiedassessmentreport.

B.Non-FunctionalEssentialities

Nonfunctionalessentialitiesarethosethatdonotdirectlyrelatetothesystem'sperformanceofthegivenfunction.They mightbeconnectedtoemergentsystemattributesincludingdependability,responsetime,andstoreoccupancy.

C.HARDWAREANDSOFTWARENECESSITIESHARDWARE

Figure6.

6.RESULTS

Considerableobjectivesalongthisworkbeingdesigninganefficient,accuratealgorithm,thusidentifiesattitudeanalysis alongtheinterviewaspirants,attitudeidentificationofaspirantsSupportsaspirantswhoareunabletoparticipatein interviewoncompanyvicinityBenefitsalongsavingtime,manpowerofinterviewer.Alongfacialidentificationto functioncapably,itsnecessarytoavailanimaginginputthatshouldnotblur/printedHereadaptedalgorithmalongfacial identification,featureextractions,systemgeneratesautomaticquestionnairesagainstaspirantsavailablealongcomputer, alsoidentifythepersonalityofaspirantalongthemodeofansweringthequestions.Feedbackbeinggenerated automatically,thusreceivedofinterviewersmailbox.Functionalrealtimeanalysis,probabilitydatarepresentations

7.CONCLUSION

Wesuggestastressrecognitionsystemthatmakesuseoffacelandmarksandfacephotos.Theexperiment'sfindings showedthatemployingfacelandmarksenhancedtheperformanceofstressrecognition.Becausetheymakeiteasierfor youtocomprehendeye,mouth,andheadmovements,faciallandmarksarebetterathelpingyoudetectstress.Wealso discoveredthatwhenemployingagreyfacialimageoftherightsize,performancewasenhancedbymoreaccurately identifyingstress-relatedinformation.

www.iaset.us editor@iaset.us

BasedAutomaticPersonalityRecognitionUsedinAsynchronousVideoInterviewsofStressDetectionAlgorithm 7

Futurestudiesaimtoenhancetheperformanceofstressrecognitionbyutilisingtime-axisinformationonhead, mouth,andeyemotion.

REFERENCES

1.Sriramprakash,S.,Prasanna,V.D.,&Murthy,O.R.(2017).Stressdetectioninworkingpeople.Procedia computerscience,115,359-366.

2.Mohd,M.N.H.,Kashima,M.,Sato,K.,&Watanabe,M.(2015).Mentalstressrecognitionbasedonnon-invasive andnon-contactmeasurementfromstereothermalandvisiblesensors.InternationalJournalofAffective Engineering,14(1),9-17.

3.Prasetio,B.H.,Tamura,H.,&Tanno,K.(2018,May).SupportVectorSlantBinaryTreeArchitectureforFacial StressRecognitionBasedonGaborandHOGFeatureIn2018InternationalWorkshoponBigDataand InformationSecurity(IWBIS)(pp.63-68).IEEE.

4.Zhang,K.,Zhang,Z.,Li,Z.,&Qiao,Y.(2016).Jointfacedetectionandalignmentusingmultitaskcascaded convolutionalnetworks.IEEESignalProcessingLetters,23(10),1499-1503.

5.Kazemi,V.,&Sullivan,J.(2014).Onemillisecondfacealignmentwithanensembleofregressiontrees.In ProceedingsoftheIEEEconferenceoncomputervisionandpatternrecognition(pp.1867-1874).

6.He,K.,Zhang,X.,Ren,S.,&Sun,J.(2016).Deepresiduallearningforimagerecognition.InProceedingsofthe IEEEconferenceoncomputervisionandpatternrecognition(pp.770-778).

7.Nishanth,N.,AndKamalKhumarLs."AnAutomaticPersonalityRecognitionSystemUsingTimeDistributed Cnn."InternationalJournalofComputerScienceEngineeringandInformationTechnologyResearch(IJCSEITR)

ISSN(P):2249

6831;ISSN(E):2249

7943Vol.12,Issue1,Jun2022,1-12

8 PriyankaH&BLJayakumar
ImpactFactor(JCC):8.5226 NASRating:3.17

Turn static files into dynamic content formats.

Create a flipbook
Issuu converts static files into: digital portfolios, online yearbooks, online catalogs, digital photo albums and more. Sign up and create your flipbook.