Image processing apparatus for displaying an analysis result of a multi-cropping scan processing转让专利
申请号 : US16778017
文献号 : US11412103B2
文献日 : 2022-08-09
发明人 : Daijiro Miyamoto
申请人 : CANON KABUSHIKI KAISHA
摘要 :
权利要求 :
What is claimed is:
说明书 :
The technology of the present disclosure relates to a technology for displaying document image data, which are generated by scanning multiple documents, in an organized manner.
Conventionally, there has been multi-cropping processing in which multiple documents such as receipts, business cards, or various kinds of certificates are arranged and placed on a document positioning plate of a scanner apparatus, so as to generate document image data from the obtained scanned image data by cropping image areas corresponding to the respective documents. In this processing, document detection processing is performed on scanned image data, which is image data corresponding to the entire document positioning plate, and an image area corresponding to each of the detected documents is cropped, so as to generate document image data.
Japanese Patent Laid-Open No. 2018-74180 discloses a technology for determining the order of pages of multiple document image data cropped by multi-cropping processing and converting the multiple document image data into one file. In Japanese Patent Laid-Open No. 2018-74180, the order of pages is determined by use of position coordinates, dimensions, similarity levels between images, or the like, of regions corresponding to originals detected by multi-cropping processing. Furthermore, the cropped document image data are displayed in order according to the determined order of pages, and a UI screen is displayed to allow the user to check the order and change the order.
However, according to the technique disclosed in Japanese Patent Laid-Open No. 2018-74180, multiple document images are simply displayed in the determined page order. Therefore, there is an issue that it is difficult to grasp, only from the display of the UI screen, which of the documents placed on the document positioning plate each of the displayed document images corresponds to.
The technology of the present disclosure includes: acquiring position information of each of a plurality of document image areas, which are identified by multi-cropping processing performed on scanned image data, and information of an analyzing result obtained by analysis of document image data corresponding to each of the plurality of document image areas; and controlling the information of the analyzing result corresponding to one of the plurality of document image areas to be displayed together with the scanned image data on a display apparatus, wherein the scanned image data is displayed in such a manner that the one of the plurality of document image areas corresponding to the information of the analyzing result being displayed is identifiable.
Further features of the present invention will become apparent from the following description of exemplary embodiments with reference to the attached drawings.
Hereinafter, an explanation is given of an embodiment for implementing the technology of the present disclosure with reference to the drawings. Noted that the following embodiment does not limit the invention according to the scope of the patent claims and that every combination of the characteristics explained in the embodiment is not essential to the solution means of the invention.
<Overall Configuration>
An MFP (Multifunction Peripheral) 110 is a multifunction peripheral having multiple functions, such as a scanner and a printer, and is an example of an image processing apparatus.
The MFP cooperative service 120 is an example of a service having a function of saving an image file obtained by scanning by the MFP 110 on its own server and a function of transferring the image file to a service that is capable of saving a file, such as another storage service.
The expense accounting service 130 is a service that provides a function for a company to efficiently perform expense accounting. In the expense accounting service 130, it is possible to save and manage image data of a scanned receipt and metadata, which is additional information of the receipt, and to set operator information of a proxy applicant who scans a receipt on behalf of the actual applicant (user to be the processing target), for example.
Although the image processing system according to the present embodiment is configured with the MFP 110 and the MFP cooperative service 120, the image processing system according to the present embodiment is not limited to the embodiment. For example, the MFP 110 may also have a role of the MFP cooperative service 120. Furthermore, there may be such a connection form in which the MFP cooperative service 120 is arranged in a server on a LAN, not on the Internet.
<Hardware Configuration of MFP>
The operation unit I/F 215 is an interface that connects the operation unit 220 and the control unit 210. The operation unit 220 includes a touch panel, a keyboard, and the like, and accepts an operation, an input, or an instruction by the user. The printer I/F 216 is an interface that connects the printer unit 221 and the control unit 210. Image data for printing is transferred from the control unit 210 to the printer unit 221 via the printer I/F 216 and printed on a printing medium such as a printing sheet. The scanner I/F 217 is an interface that connects the scanner unit 222 and the control unit 210. The scanner unit 222 reads a document that is set on a document positioning plate or an ADF (Auto Document Feeder), each of which is not illustrated in the drawings, and generates image data, in order to input the image data to the control unit 210 via the scanner I/F 217. The MFP 110 is capable of printing (copying) image data generated by the scanner unit 222 by use of the printer unit 221 as well as performing file transmission or email transmission of the image data. The modem I/F 218 is an interface that connects the modem 223 and the control unit 210. The modem 223 is capable of performing facsimile communication of image data with a facsimile apparatus on PSTN. The network I/F 219 is an interface that connects the control unit 210 (MFP 110) to a LAN. The MFP 110 is capable of receiving various types of information and transmitting image data and information to a service on the Internet by use of the network I/F 219.
<Hardware Configuration of MFP Cooperative Service and Expense Accounting Service>
The MFP cooperative service 120 and the expense accounting service 130 are configured with a CPU 311, a ROM 312, a RAM 313, an HDD 314, and a network I/F 315. The CPU 311 controls the overall operation by reading out a control program stored in the ROM 312 and executing various kinds of processing. The RAM 313 is used as a temporary memory area such as a main memory or work area of the CPU 311. The HDD 314 is a large-capacity storage unit that stores image data and various programs. The network I/F 315 is an interface for connecting the MFP cooperative service 120 and the expense accounting service 130 to the Internet. The MFP cooperative service 120 and the expense accounting service 130 are capable of transmitting and receiving various kinds of information in response to a processing request received from another apparatus (such as the MFP 110) via the network I/F 315.
<Software Configuration of Image Processing System>
The standard function unit 410 includes a scan execution unit 411 and an image data storage unit 412. The additional function unit 420 includes a display control unit 421, a scan designation portion 422, and a cooperative service request unit 423.
The display control unit 421 displays a UI screen for accepting an operation input by the user, such as the UI screen represented in
The scan execution unit 411 receives a scan request including a scan setting from the scan designation portion 422. In accordance with the scan request, the scan execution unit 411 reads a document placed on the document positioning plate made of glass by use of the scanner unit 222 via the scanner I/F 217, in order to generate scanned image data, which is image data of the entire document positioning plate. The generated scanned image data is transmitted to the image data storage unit 412. The scan execution unit 411 transmits a scanned image identifier, which uniquely represents scanned image data that is saved, to the scan designation portion 422. The image data storage unit 412 stores the scanned image data received from the scan execution unit 411 in the HDD 214.
The scan designation portion 422 acquires, from the image data storage unit 412, the scanned image data corresponding to the scanned image identifier received from the scan execution unit 411. The scan designation portion 422 requests the cooperative service request unit 423 for an instruction to perform multi-cropping processing on the acquired scanned image data.
The cooperative service request unit 423 requests the MFP cooperative service 120 for various kinds of processing. For example, the cooperative service request unit 423 requests for login, acquisition of information of a proxy application eligible person (operator information) who is eligible for expense accounting by proxy, multi-cropping processing, acquisition of metadata information, setting of metadata, or the like. Although a protocol such as REST or SOAP is used for communication with the MFP cooperative service 120, other communication means may be used.
The MFP cooperative service 120 includes a request control unit 431, an image processing unit 432, an expense accounting service request unit 433, a data management unit 434, and a display control unit 435.
The request control unit 431 stands by in a status of being able to receive a request from an external apparatus. Upon receiving a processing request, the request control unit 431 provides an instruction for processing to the image processing unit 432, the expense accounting service request unit 433, or the data management unit 434 as appropriate according to the request.
The image processing unit 432 is an image acquisition unit that performs multi-cropping processing or image analysis processing, such as OCR or layout analysis, on image data, so as to acquire document image data.
The expense accounting service request unit 433 requests the expense accounting service 130 for various kinds of processing. For example, the expense accounting service request unit 433 requests for acquisition of a list of proxy application eligible persons, acquisition of a list of expense types, registration of image data of a receipt, and setting of metadata corresponding to image data of a receipt.
The data management unit 434 holds user information, various kinds of setting data, etc., that are managed by the MFP cooperative service 120.
The display control unit 435 returns screen layout information (HTML, CSS, etc.) that is necessary for screen display in response to a request from a web browser operating on another terminal, which is not illustrated in the drawings, such as a PC or a mobile connected via the Internet. The user checks user information registered in the MFP cooperative service 120 or changes settings for scanning, etc., via the screen displayed on the web browser.
The expense accounting service 130 includes a request control unit 441, an expense information management unit 442, and a display control unit 443.
The request control unit 441 stands by in a status of being able to receive a request from an external apparatus. Upon receiving a processing request, the request control unit 441 provides an instruction for processing to the expense information management unit 442 as appropriate according to the request.
The expense information management unit 442 holds expense information registered in the expense accounting service 130.
The display control unit 443 returns screen layout information (HTML, CSS, etc.) that is necessary for screen display in response to a request from a web browser operating on another terminal, which is not illustrated in the drawings, such as a PC or a mobile connected via the Internet. Via the screen displayed on the web browser, it is possible for the user to check expense information, which includes image data of a receipt and information of an expense report registered in the expense accounting service 130, and to input data, etc.
<Overall Processing Flow>
In a normal status, the MFP 110 displays, on a touch panel, a main screen in which buttons for performing respective functions to be provided are arranged.
In S501, the expense accounting application displays a login screen on which authentication information for accessing the MFP cooperative service 120 is input.
In a case where the “LOGIN” button 701 is pressed by the user, the expense accounting application transmits the login request to the MFP cooperative service 120 in S502.
In S503, the MFP cooperative service 120 verifies whether the user name and password included in the login request are correct or not and returns an access token to the MFP 110 in a case where the user name and password are correct. Hereinafter, various kings of requests provided from the MFP 110 to the MFP cooperative service 120 are issued together with the above-described access token, so that the user to be the processing target can be identified by use of the access token. As for the method for performing user authentication, a publicly known method (Basic authentication, Digest authentication, authorization by use of OAuth, etc.) in general may be used.
In S504, upon completion of the login processing and receiving the access token, the MFP 110 transmits an acquisition request for proxy application eligible person information to the MFP cooperative service 120. For the request to the MFP cooperative service 120, it is possible to use a protocol such as REST.
In S505, upon receiving the acquisition request for the proxy application eligible person information, the MFP cooperative service 120 transmits an acquisition request for the proxy application eligible person information to the expense accounting service 130 by use of the access token, which is associated with the user of the expense accounting service 130. In the MFP cooperative service 120, the user thereof is associated with the user of the expense accounting service 130 in advance by use of a web browser operating on a PC, etc., that is connected via the Internet. Therefore, it is possible for the expense accounting service 130 to uniquely identify the user of the expense accounting service 130 from among the users of the MFP cooperative service 120. Accordingly, upon receiving the acquisition request for the proxy application eligible person information, the expense accounting service 130 returns, to the MFP cooperative service 120, information of the proxy application eligible person who is set for the user of the expense accounting service 130 corresponding to the user of the MFP cooperative service 120.
In S506, the MFP 110 displays a proxy application screen based on the acquired proxy application eligible person information.
In S507, the MFP 110 displays a scan setting screen.
In S508, the MFP 110 generates scanned image data, which is image data of the entire document positioning plate that has been read by execution of scanning.
In S509, the MFP 110 transmits an analysis request of the scanned image data to the MFP cooperative service 120.
In S510, upon receiving the analysis request of the scanned image data, the MFP cooperative service 120 starts multi-cropping processing by use of the image processing unit 432 of the MFP cooperative service 120. Thereafter, the MFP cooperative service 120 returns a request ID that uniquely represents the analysis request of the scanned image data to the MFP 110 without waiting for the end of the multi-cropping processing.
After receiving the analysis request of the scanned image data, in a case where a document is detected as a result of the multi-cropping processing by the image processing unit 432, the MFP cooperative service 120 performs a rotation process, a character recognition (OCR) process, and document type detection on each detected document. In the rotation process, in a case where the read document is inclined relative to a predetermined reference line, the document is corrected such that an edge of the document becomes parallel or perpendicular to the reference line. Further, by detecting the contents thereof, the document is rotated in a direction to make the text of the document be in the correct orientation. Then, character recognition and contents detection are performed, so as to extract a character string of the document and detect the type of the document.
In S511 to S513, the MFP 110 periodically (for example, every 3 seconds) checks the processing status of the image analysis with the MFP cooperative service 120 by use of the request ID received in S510. Although omitted in
The response upon completion of the processing includes information about the result of analysis of the scanned image data as well as the status. In “Images”, information for the number of documents detected by the multi-cropping processing is stored. “ImageID” represents an ID of detected document image data, and, in a case where an acquisition request of document image data is transmitted to the MFP cooperative service 120 together with designation of an Image ID, the targeted document image data can be acquired. “ContentsType” represents the type of a detected document. For example, the character string may be “Receipt” for a receipt, “BusinessCard” for a business card, and “DriverLicense” for a license. “ReceiptData” is included in a case where “ContentsType” is “Receipt”, which represents a receipt. “ReceiptData” stores values indicating a date and a total money amount, which are acquired as a result of analysis of the contents of a document. “Points” stores coordinates representing an area of document image data in scanned image data, that is, coordinates of an image area in scanned image data corresponding to document image data. From the initial position, the X coordinate of the top left point, the Y coordinate of the top left point, the X coordinate and Y coordinate of the points at positions clockwise about the center of the document from the top left point are stored in that order.
In S514, in response to receiving the response upon completion of processing, the MFP 110 determines the order of the document image data for setting metadata in the processing of S519 by use of the detected information. An explanation is given later of details of the method for determining the order of the document image data.
Subsequently, in S515, the MFP 110 transmits an acquisition request for document image data to the MFP cooperative service 120 with designation of the “ImageID” of the document image to be displayed first. Upon receiving the acquisition request for document image data, the MFP cooperative service 120 transmits the document image data corresponding to the “ImageID” to the MFP 110. The document image data to be returned here is document image data in the corrected orientation as a result of the rotation process.
Next, in S516, the MFP 110 transmits an acquisition request for item information of metadata to the MFP cooperative service 120.
In S517, upon receiving the acquisition request for metadata from the MFP 110, the MFP cooperative service 120 acquires the metadata item information for the target user from the expense accounting service 130.
In the embodiment represented in
In S518, the MFP 110 displays the metadata setting screen 1400 as represented in
In S519, upon detecting that the “TRANSMIT” button 1401 is pressed by the user, the MFP 110 transmits a metadata setting request to the MFP cooperative service 120 together with the metadata information set on the metadata setting screen 1400.
In S520, in response to receiving the metadata setting request, the MFP cooperative service 120 transmits a metadata setting request to the expense accounting service 130, based on the received metadata information. Upon receiving the metadata setting request, the expense accounting service 130 sets the received metadata information to expense information managed by itself and then returns information representing completion of the metadata setting to the MFP cooperative service 120.
Upon receiving the information representing completion of the metadata setting from the expense accounting service 130, the MFP cooperative service 120 returns a notification indicating completion of the metadata setting to the MFP 110.
Upon receiving the notification indicating completion of the metadata setting from the MFP cooperative service 120, the MFP 110 ends the series of processing.
<Details of Processing by MFP>
This processing is started in a case where it is detected that the “EXPENSE ACCOUNTING” button 601 is pressed on the main screen 600 (
In S1601, the display control unit 421 displays the login screen 700. In order to log in, the user needs to input a user name in the text input area 702, input a password in the text input area 703, and select the “LOGIN” button 701 on the login screen 700. Upon detecting that login information including the user name and the password is input by the user and the “LOGIN” button 701 is pressed, the display control unit 421 transmits a login request to the cooperative service request unit 423.
In S1602, the cooperative service request unit 423 transmits a login request to the request control unit 431 of the MFP cooperative service 120. The request control unit 431 of the MFP cooperative service 120 accesses the data management unit 434 to check whether the login information is correct or not. In a case where the login information is correct, the request control unit 431 returns an access token to the cooperative service request unit 423 of the MFP 110, and the processing jumps to S1603.
In S1603, the cooperative service request unit 423 transmits an acquisition request of proxy application eligible person information to the request control unit 431 of the MFP cooperative service 120 together with the access token. Upon receiving the acquisition request of proxy application eligible person information, the request control unit 431 of the MFP cooperative service 120 requests the expense accounting service request unit 433 for acquisition of proxy application eligible person information. Then, the expense accounting service request unit 433 acquires user information of the expense accounting service 130 associated with the access token. Further, the expense accounting service request unit 433 transmits an acquisition request of the proxy application eligible person information of the target user to the request control unit 441 of the expense accounting service 130. Upon receiving the acquisition request of the proxy application eligible person information, the request control unit 441 of the expense accounting service 130 acquires the proxy application eligible person information of the target user from the expense information management unit 442 and returns the proxy application eligible person information to the expense accounting service request unit 433. Upon receiving the proxy application eligible person information, the expense accounting service request unit 433 returns the proxy application eligible person information to the cooperative service request unit 423 of the MFP 110 through the request control unit 431.
Upon receiving the proxy application eligible person information in S1604, the cooperative service request unit 423 of the MFP 110 transmits the proxy application eligible person information to the display control unit 421. Further, the display control unit 421 displays the proxy application screen 900. It is possible that the processing of S1604 is skipped and the processing of S1605 is executed in a case where there is no proxy application eligible person according to the proxy application eligible person information acquired in the processing of S1603. On the proxy application screen 900, the user needs to select a proxy application eligible person to be used from the proxy application eligible person list 902 and select the “NEXT” button 901 in order to determine a proxy application eligible person. It is possible that a value representing the actual applicant is set in the initial state of the proxy application eligible person list 902 and, in a case where the value is not changed, such processing in which document image data of a document scanned by the actual applicant is registered in the expense accounting service 130 is performed.
In S1605, in a case where the “NEXT” button 901 is pressed, the display control unit 421 displays the scan setting screen 1000.
In S1606, upon detecting that the “START SCANNING” button 1001 is pressed by the user, the display control unit 421 provides an instruction for scanning to the scan designation portion 422. The scan designation portion 422 instructs the scan execution unit 411 for scanning, so that scanning of the document is performed. The scanned image data generated by scanning is saved in the image data storage unit 412, and an identifier corresponding to the scanned image data is notified to the scan designation portion 422. In S1606, a screen (not illustrated in the drawings) representing that scanning is in progress may be displayed.
Next, in S1607, the cooperative service request unit 423 transmits the scanned image data acquired via the scan designation portion 422 to the request control unit 431 of the MFP cooperative service 120 together with an analysis request of the scanned image. The request control unit 431 extracts the scanned image data from the analysis request of the scanned image and transmits the scanned image data to the image processing unit 432, so that multi-cropping processing is executed. Upon completion of the multi-cropping processing by the image processing unit 432, the request control unit 431 subsequently makes the image processing unit 432 perform an image rotation process, a document type detection process, and a character recognition process. Upon starting the multi-cropping processing, the request control unit 431 immediately returns a request ID to the cooperative service request unit 423 of the MFP 110 without waiting for completion of the processing. The request ID is a value by which an analysis request of a scanned image to be processed in the MFP cooperative service 120 can be uniquely identified.
In S1608, upon acquiring the request ID, the cooperative service request unit 423 transmits an acquisition request of a processing status together with the request ID to the request control unit 431 of the MFP cooperative service 120. The request control unit 431 queries the image processing unit 432 for the status of image processing corresponding to the request ID received together with the acquisition request of the processing status. In a case where the processing corresponding to the request ID is being under processing, the image processing unit 432 returns information representing being under processing. In a case where the processing is completed, the image processing unit 432 returns information representing completion, processing results of multi-cropping (position information of each document as well as image data of each document), and analyzing results of various types of images (a determination result of the document type of each document image as well as a result of the character recognition process) to the request control unit 431. The request control unit 431 creates response data for the acquisition request of the processing status, based on the information processed by the image processing unit 432, and returns the response data to the cooperative service request unit 423 of the MFP 110.
In S1609, the cooperative service request unit 423 determines whether the acquired processing status is a value representing being under processing or not. In a case where the processing status is a value representing being under processing, the processing returns to S1608 after a few seconds or so, so that the cooperative service request unit 423 performs the processing of transmitting the request of checking the processing status again. In a case where the processing status is a value representing completion, the cooperative service request unit 423 acquires analysis information included in the response to the request of checking the processing status.
In S1610, the cooperative service request unit 423 transmits an acquisition request of metadata item information to the request control unit 431 of the MFP cooperative service 120 together with the access token. Upon receiving the acquisition request of metadata item information, the request control unit 431 requests the expense accounting service request unit 433 to acquire metadata item information. Then, the expense accounting service request unit 433 acquires the user information of the expense accounting service 130 associated with the access token and transmits an acquisition request of the metadata item information corresponding to the target user to the request control unit 441 of the expense accounting service 130. Upon receiving the request, the request control unit 441 acquires the metadata item information from the expense information management unit 442 and returns the metadata item information to the expense accounting service request unit 433 of the MFP cooperative service 120. Upon receiving the metadata item information, the expense accounting service request unit 433 returns the metadata item information to the cooperative service request unit 423 of the MFP 110 through the request control unit 431.
In S1611, the display control unit 421 of the MFP 110 determines the display order of each document image data, based on the response data acquired in S1608.
In S1701, the display control unit 421 of the MFP 110 determines whether a proxy application eligible person is selected in S1604 or not. In a case where a person other than the actual applicant makes an application by proxy, the processing proceeds to S1703, in which document image data are rearranged so as to be displayed in order from the document image data having the largest X coordinate of the upper right corner of the image area, which is the corresponding position information in the scanned image data. The position information including the coordinates can be acquired from the response data acquired in S1608. In the case of the response data in
In a case of determining that the actual applicant is selected in S1701, the display control unit 421 determines whether the document image data includes information related to a receipt or not in S1702. Whether document image data corresponds to a receipt or not can be acquired from the response data acquired in S1608. In the case of the response data in
In S1704, in a case where it is determined that a document corresponding to document image data is a receipt, the display control unit 421 rearranges the document image data so as to be displayed in order from one with the oldest date. Date information including the date can be acquired from the response data acquired in S1608. In the case of the response data in
Returning to
The expense type selection list 1404 is a list for selecting the expense type of the receipt corresponding to the document image data to be the setting target, which is displayed in the document image display area 1410. The expense type list displayed in the document image display area 1410 is based on the information acquired in S1610. In the example of
The scanned image display area 1408 is an area in which the entire scanned image is displayed in a reduced size. If the scanned image data is displayed as it is, a document placed on the left side from the view of the operator who placed the document is displayed on the right side. Therefore, an image in a position relationship with the left and right inversed is displayed. However, if the scanned image data is displayed with the left and right simply inversed, the contents of the document are also displayed in the inversed manner. Therefore, for example, the left and right may be inversed again within each of the image areas corresponding to the documents. With such processing, it is possible to display an image of which a position relationship among image areas corresponding to documents of scanned image data is left-right inversed but each of the image areas corresponding to the documents is not left-right inversed.
Furthermore, in the present embodiment, numbers representing the display order are superimposed on the displayed image of the scanned image data and displayed at the locations corresponding to the documents. Further, the image area corresponding to the document image data that is currently displayed in the document image display area 1410 is displayed with highlighting so as to be identifiable. In addition, it is possible to change display of an image area corresponding to document image data to which metadata has already been set, such that it is represented that processing has already been done. Furthermore, by selecting a document image on the scanned image display area 1408, the document image can be displayed in the document image display area 1410.
A display order selection list 1409 is a list for selecting the display order of multiple document image data generated by multi-cropping processing. It is possible that the order is changed over to various orders, such as the date order and arrangement order. In a case where the order is changed, the document image data in the initial position in the changed order is displayed, and the order displayed in the scanned image display area 1408 is also changed over to the changed order. For example, in a case where the arrangement order (from the left) is selected in the display order selection list 1409 in the display form illustrated in
In S1613, the display control unit 421 determines the button selected on the metadata setting screen 1400.
In a case where the display control unit 421 detects that the “NEXT” button 1402 is pressed in S1613, the processing jumps to S1614. Then, the display control unit 421 targets the next document image data of the document image data being displayed and returns to S1612 to display a metadata setting screen for the targeted document image data.
In a case where the display control unit 421 detects that the “RETURN” button 1403 his pressed in S1613, the processing jumps to S1615. Then, the display control unit 421 targets the previous document image data of the document image data being displayed and returns to S1612 to display a metadata setting screen for the targeted document image data.
In a case where the display control unit 421 detects that the “TRANSMIT” button 1401 is pressed in S1613, the processing jumps to S1616. Then, the cooperative service request unit 423 transmits, to the request control unit 431, a metadata setting request for setting the contents set on the metadata setting screen for the corresponding document image data, respectively.
Upon receiving the metadata setting request, the request control unit 431 requests the expense accounting service request unit 433 for setting of metadata. Then, the expense accounting service request unit 433 acquires the user information of the expense accounting service 130 associated with the access token and transmits a metadata setting request for the target user to the request control unit 441 of the expense accounting service 130. Upon receiving the request, the request control unit 441 of the expense accounting service 130 performs, on the expense information management unit 442, setting of metadata information of the receipt for the document image data corresponding to “ImageID” included in the request. Upon receiving the information, the expense accounting service request unit 433 returns information representing completion of the setting to the cooperative service request unit 423 of the MFP 110 through the request control unit 431. Upon receiving the information representing completion of the setting from the MFP cooperative service 120, the cooperative service request unit 423 ends the processing.
<Application to Other Services>
In the above-explained example, the document image data of a scanned receipt and metadata corresponding to the document image data are set to the expense accounting service 130. However, it is also possible to adapt the present embodiment for a case of cooperating with another service. For example, such a configuration as the metadata setting screen suggested in the present embodiment may be adapted for a case of applying to a business card management service, etc. As for business cards, operation may be easier in a case where business cards are displayed in order for each group of people who belong to the same company. Therefore, it is also possible that the display order of document image data is determined based on a similarity level search of document image data or layout information of the included contents, which is obtained in such document analysis processing by the MFP cooperative service 120 as explained in S512.
<Operation when Scanning Multiple Sheets>
In the above-explained example, the explanation is given with an operation in a case where scanning is performed once. However, it is also possible that scanning is performed multiple times and that scanning is performed one by one for multiple receipts in a standard size such as A4 by use of an ADF (Auto Document Feeder), or the like. For example, in a case of scanning multiple receipts arranged on the document positioning plate multiple times, an item for setting scanning multiple times is provided on the scan setting screen 1000 (
Similarly, in a case where scanning is performed one by one from the ADF, analysis is performed for all of the scanned image data after scanning all of the documents. Here, since multi-cropping processing is not necessary, it is also possible that such a parameter indicating not to perform multi-cropping in a case of scanning from the ADF is added to the request, so as to notify the MFP cooperative service 120 that multi-cropping processing is not required. In a case of scanning from the ADF, display on the scanned image display area 1408 is not necessary. Therefore, alternatively, it is also possible that the document image data for which metadata is being set and the document image data generated by scanning before and after the document image data for which metadata is being set are displayed in an arranged manner. Furthermore, it is also possible to display which number of scanned document each document image data corresponds to.
<Setting of Display Order>
The display order in the initial state may be changed on a display order setting screen that is separately provided. Setting on a display order setting screen can be performed through an icon button, or the like, which is provided somewhere on the metadata setting screen for displaying the display order setting screen, for example.
The order determination information selection list 1902 is a list for selecting order determination information for determining the display order in a case of determining the display order. For example, the order determination information includes “OPERATOR INFORMATION” and “DOCUMENT TYPE”. In a case where “OPERATOR INFORMATION” is selected, the order is determined by use of operator information. In a case where “DOCUMENT TYPE” is selected, the order is determined by use of document type information. In a case where “OPERATOR INFORMATION” is selected in the order determination information selection list 1902, the display order is determined by use of the contents set in the lists 1903 and 1904. In a case where the operator is the actual applicant, the order is determined by the method set in the list 1903. Further, in a case where the proxy application processing is performed by a person other than the actual applicant, the order is determined by the method set in the list 1904.
According to the setting in the status of
<Function Assignation for Each Apparatus>
In the explanation of the above-described embodiment, the MFP 110, the MFP cooperative service 120, and the expense accounting service 130 cooperate with each other in order to realize the functions. However, another configuration may be employed. For example, the MFP 110 may have the function of the MFP cooperative service 120, so that the configuration only includes the MFP 110 and the expense accounting service 130.
(Other Embodiments)
Embodiment(s) of the present invention can also be realized by a computer of a system or apparatus that reads out and executes computer executable instructions (e.g., one or more programs) recorded on a storage medium (which may also be referred to more fully as a ‘non-transitory computer-readable storage medium’) to perform the functions of one or more of the above-described embodiment(s) and/or that includes one or more circuits (e.g., application specific integrated circuit (ASIC)) for performing the functions of one or more of the above-described embodiment(s), and by a method performed by the computer of the system or apparatus by, for example, reading out and executing the computer executable instructions from the storage medium to perform the functions of one or more of the above-described embodiment(s) and/or controlling the one or more circuits to perform the functions of one or more of the above-described embodiment(s). The computer may comprise one or more processors (e.g., central processing unit (CPU), micro processing unit (MPU)) and may include a network of separate computers or separate processors to read out and execute the computer executable instructions. The computer executable instructions may be provided to the computer, for example, from a network or the storage medium. The storage medium may include, for example, one or more of a hard disk, a random-access memory (RAM), a read only memory (ROM), a storage of distributed computing systems, an optical disk (such as a compact disc (CD), digital versatile disc (DVD), or Blu-ray Disc (BD)™), a flash memory device, a memory card, and the like.
While the present invention has been described with reference to exemplary embodiments, it is to be understood that the invention is not limited to the disclosed exemplary embodiments. The scope of the following claims is to be accorded the broadest interpretation so as to encompass all such modifications and equivalent structures and functions.
According to the technology of the present disclosure, in a case where multiple documents are placed on a document positioning plate, it is possible to display document images on a UI screen in such a manner that the user can easily check the correspondence relationship between the document images being displayed on the UI screen and the corresponding documents placed on the document positioning plate.
This application claims the benefit of Japanese Patent Application No. 2019-023573 filed Feb. 13, 2019, which is hereby incorporated by reference wherein in its entirety.